15 — Recommendations
This section is the normative voice of the manual. The previous sections describe what Actual Vote is, how it works, what AC has done, and why AC does it the way it does. This section says what AC believes the world should do, based on what AC has learned from its work.
The recommendations are AC’s, not anyone else’s. They are based on the empirical experience documented in Section 8 (methodology), Section 10 (operational practice), Section 12 (findings), Section 13 (philosophy), and the Case Studies section. They are presented as positions held by AC, not as objective truths or as widely-accepted standards. A reader who disagrees is welcome to disagree; AC is presenting its position and inviting engagement, not declaring policy.
A note on the structure of this section. Section 15 may eventually break off into its own standalone document — a “Gold Standard” or “AC Policy Recommendations” paper that exists separately from the technical manual. The reasons for the eventual split: the rest of the manual is descriptive and is likely to be cited by readers who have no interest in AC’s normative positions; the recommendations are normative and may benefit from being discussed separately. For this edition of the manual, the recommendations live here; future editions may move them out.
The recommendations are aimed at one overarching goal: trustworthy vote reporting. “Trustworthy” is more demanding than “accurate.” A vote reporting system can be accurate without being trustworthy, if the public has no way to verify the accuracy. The combination — accurate and visibly accurate, in ways that independent parties can verify — is trustworthy. Every recommendation in this section can be traced back to that goal.
The recommendations are not radical critiques of the existing system. AC’s experience is that most US elections offices are doing their best to run accurate elections with the resources and constraints they have. The recommendations are not “the system is fundamentally broken.” They are “the system works reasonably well, and here are specific things that would make it work better.”
This framing matters because it shapes how the recommendations are likely to be received. Elections offices that hear “you are part of the problem” tend to become defensive. Elections offices that hear “here are specific operational improvements that would help everyone” tend to be more receptive. AC’s recommendations are designed to be in the latter category.
This doesn’t mean AC has no critique. The recommendations include some things that would require real changes in how some offices operate. The Wake County NC PDF obfuscation case, for example, is a clear example of an office whose practices make external verification harder, and AC’s recommendations would push back on that. But the critique is specific and operational, not sweeping and ideological.
Each recommendation in this section is grounded in something AC has actually encountered in its operational work. AC isn’t recommending things in the abstract; AC is recommending things based on having seen them work or seen their absence cause problems. The grounding gives the recommendations a different character than typical policy proposals. They are smaller in scope, more concrete, and more specifically actionable. They are also more limited: AC isn’t recommending things AC hasn’t seen the effects of. Where AC has opinions about things outside its direct experience, those opinions are flagged as opinions rather than presented as recommendations. Section 9 (Philosophy) is the place for AC’s broader views; this section is for specific operational suggestions.
There are four subsections. Section 15.1 addresses elections offices. Section 15.2 addresses election system designers and legislators. Section 15.3 addresses voters and the broader civic ecosystem. Section 15.4, new to this edition, addresses data science and measurement — how the impact of vote reporting verification can be quantified (through metrics like margin proximity and coverage ratio), which appealing-sounding metrics turn out not to be useful, and how AV operations can be targeted for maximum effectiveness.
A reader who is part of one of these audiences — an elections office staffer, a voting equipment vendor, an engaged citizen — should read the relevant subsection. A reader interested in AC’s overall normative posture should read all four.
15.1 — Recommendations to Elections Offices
Elections offices are AC’s most important audience for recommendations because they directly administer elections and report results. Changes in elections office practices have the most immediate effect on the trustworthiness of vote reporting. The recommendations in this subsection are based on AC’s experience working with various offices over the course of many analyses. They are organized by topic.
1. Post poll tapes publicly after polls close
When polls close, election workers should post a copy of each polling place’s poll tapes in a publicly visible location at the polling place. The tapes should remain posted for a reasonable period — at least overnight and ideally until the next morning — so that members of the public can view and (legally) record them.
Public posting is the foundation of Method 1 (Section 7.2) and the cheapest, simplest form of external verification. Jurisdictions that post tapes enable anyone — not just AC, but any citizen — to independently check the numbers. The Broward 2018 experience documented one-third posting non-compliance even in a jurisdiction with clear posting requirements.
The operational details matter. Tapes should be posted in a publicly accessible location at or near the polling place — a wall, window, or door visible from public space (the entrance, the parking lot, the sidewalk). If the polling place is inside a building with restricted public access after hours, the tapes should be posted somewhere that does not require entering the restricted area. Each machine’s tape should be posted in its entirety, not folded or trimmed in ways that hide values. Tapes should be posted right-side up and unfolded, so that values are clearly visible to anyone trying to read them. They should remain posted at least until the next morning, and ideally until the polling place is being prepared for its next election day use. Many jurisdictions remove tapes within an hour or two after closing; this is shorter than necessary and reduces opportunities for verification.
AC’s recommendation is that all states require public posting and that the requirements be specific enough to be enforceable: what to post, where to post it, how to orient it, and how long to leave it posted. State-level posting requirements currently vary widely: some states require posting, some permit it but do not require it, and some are silent on the question.
Wake County, NC is a jurisdiction where AC has had difficulty obtaining poll tapes through Method 1 because of variability in posting practices across polling places. Some Wake polling places post; others don’t. The variability makes Method 1 less reliable in Wake than in jurisdictions with more consistent practices. AC’s recommendation to Wake County and to similar jurisdictions: standardize posting practices so that every polling place consistently posts every poll tape after every election.
2. Report official results in machine-readable formats
Elections offices should publish official results in formats that support machine processing. CSV is the gold standard. Structured Excel (.xlsx) is acceptable. JSON or XML are also workable. PDFs are acceptable only if they are searchable text (not image-based) and are not deliberately obfuscated. PDFs should not be the primary format — they are designed for visual presentation, not for data interchange, and are difficult to process programmatically. If PDFs are published at all, they should be supplements to a structured format, not the only available form.
Independent verification depends on being able to take official results and compare them programmatically against poll tape data. When results are published only in formats that resist machine reading, the verification work becomes much slower and more expensive — which means less verification, not more. The Wake County NC 2022 experience is the canonical negative example: official results PDFs containing nuisance characters, encoding tricks, hidden columns, and font substitutions that appear designed to defeat machine extraction.
Results should be published at the precinct × voting method × contest × choice level. Aggregating up to higher levels (precinct totals only, county totals only) loses the granularity needed for comparison. If PDFs must be used, they should be searchable-text PDFs (where the text is actually text, not images) with standard character encodings. Simple, well-formed PDFs are workable; obfuscated PDFs are not. The specific anti-patterns to avoid include non-standard character encodings, hidden positioning offsets, image-substituted text, and scattered nuisance characters. AC’s recommendation: publish results in CSV alongside whatever other formats the office currently uses. The cost is minimal; the benefit to verification is substantial.
3. Provide clear discrepancy-handling procedures
Elections offices should have a documented, public process for receiving, investigating, and responding to apparent discrepancies raised by external observers (including AC, partisan poll watchers, individual citizens, and other organizations). The process should be timely, transparent, and consistent. It should specify: who to contact, what information to provide, what timeline to expect for a response, and what happens if the discrepancy is confirmed.
AC’s experience is that many offices have no established process for handling external discrepancy reports. When AC raises an apparent discrepancy with the office (a heavier ask, per the process in Section 8.8), the office may not know who should receive the communication, how to evaluate it, or what to do next. Without a clear process, the response is ad-hoc — depending on who happens to receive the inquiry, how busy they are, and what their personal disposition is. Ad-hoc responses are uneven and sometimes unsatisfactory. A clear process benefits both the office and the external observer: it sets expectations, reduces friction, and ensures that legitimate findings receive appropriate attention.
The process should specify that a designated contact person (or role) receives external inquiries and is findable from the office’s public website. Inquiries should be acknowledged within a few business days, even if a substantive response will take longer. The office should investigate substantively — not dismiss reports without examination — and respond transparently with specific enough explanations that the external party can verify the resolution. If investigation reveals that the office’s records are incorrect, the office should correct the records publicly. A log of external inquiries and their dispositions should be maintained for both internal accountability and public transparency.
The Wake County BOE response to AC’s heavier ask about an apparent early voting discrepancy in November 2024 is the positive example: a substantive investigation, a clear explanation (the ballot jam re-scan documented in Section 8.9), a timely response, and a strengthened working relationship. This is the model AC recommends to other offices.
4. Cooperate with external verification efforts
Elections offices should treat external auditors (including AC and similar organizations) as partners in election trust rather than as adversaries. Cooperation should be the default; refusal should be the exception, and exceptions should be documented and explained. Cooperation can take many forms: providing access to poll tapes through Method 3 (Section 7.4) or Method 4 (Section 7.5), responding to discrepancy reports promptly, making official results available in accessible formats, and being transparent about vote reporting procedures.
Cooperation does not mean surrendering authority. The elections office remains the entity that runs the election, counts the votes, and certifies the results. Independent verification supplements that authority; it does not challenge it. An elections office that cooperates with external verification demonstrates confidence in its own work — and gets the benefit of an independent check that may catch errors the office missed. The cost to the office is small (some staff time, some materials); the benefit (additional independent verification, evidence of transparency, support for public confidence in the office’s work) is substantial. The current pattern in which some offices cooperate willingly and others resist creates inconsistency that hurts the broader system.
When an external party requests access to public records or asks for an opportunity to observe public processes, the default response should be yes. Refusal should require a specific reason. The office should publish guidance on how external parties can access public materials, including hours, fees (if any), and required forms. Staff who handle external inquiries should be trained to be welcoming, accurate, and consistent — they should not need to make case-by-case judgments about which inquiries are legitimate. The office should distinguish between non-partisan and partisan parties: non-partisan organizations like AC are not adversaries and should not be treated as if they were. Partisan parties have a different relationship and may warrant different protocols, but the office should not treat all external parties as if they were partisan. When the office helps an external party, it should keep a record of what was provided and to whom, to protect the office from later claims that it favored one party over another.
The Citizens Audit Broward example is instructive. The Broward County FL Supervisor of Elections office cooperated substantively with the CAB Method 3 effort in 2020, allowing the team to come to the office, view poll tapes, and record them. This cooperation enabled an analysis that would not otherwise have been possible. AC’s recommendation: more offices should be like Broward in this respect.
5. Document vote reporting logic
Elections offices should publish clear documentation of how results aggregate from precinct-level poll tapes to county-level and state-level official totals, and make the documentation available to interested external parties.
As described throughout Section 8, much of the vote reporting logic in US jurisdictions is not formally documented. Election staff know how their county’s specific reporting works; outsiders generally don’t. This information asymmetry makes external verification harder and creates room for misunderstanding and mistrust. This recommendation is grounded in one of AC’s most persistent operational challenges: figuring out the comparison logic for each jurisdiction (Section 8.6). In most cases, AC has to reverse-engineer the vote reporting logic from the data itself, because the elections office has not published it. This reverse-engineering is time-consuming, error-prone, and — as documented in Section 8.7 — frequently the source of apparent discrepancies that turn out to be artifacts of AC misunderstanding the logic rather than actual reporting errors.
The basics should be documented first: how the office aggregates values from multiple machines in the same precinct, how write-ins are handled, how overvotes and undervotes are handled, how provisional ballots are integrated into final totals. Each of these has a specific answer in each jurisdiction; the answer should be written down. Then the edge cases: hurricane relocations, ballot jams, equipment failures, polling place changes — when these things happen, what does the office do? The procedures should be documented so that external observers can understand and verify them. The documentation should be available on the office’s public website, not just in internal manuals. It should be updated when procedures change, since vote reporting logic evolves over time as software is updated and as the office’s practices change.
In Section 8.6, AC notes that the vote reporting logic for each county sometimes lives in “500-page manuals” that AC has no access to. Some offices have such manuals; others have less formal documentation; others have nothing written down at all. AC’s recommendation is that all offices should aim for the documented end of this spectrum. The cost is significant (writing the documentation takes time) but the benefit is substantial: documented logic is verifiable logic, and verifiable logic is trustworthy logic. Documented vote reporting logic would eliminate a large category of false positives and make the verification process faster and more reliable for both internal and external auditors.
6. Publish clear guidance on what external observers can and cannot do
Elections offices should publish clear, accessible guidance on the rights and restrictions that apply to external observers during and after elections. This includes: what areas of the polling place are publicly accessible, what recording is permitted, what posting practices are followed, and how to request access to materials after election day. Specifically, offices should answer: are non-officials permitted to view posted poll tapes? Are they permitted to photograph or video-record posted poll tapes? Are they permitted to be present during the closing procedure? Are they permitted to record during the closing procedure?
Many of the operational difficulties AC encounters (Section 10.11) stem from ambiguity about what observers are allowed to do. Different poll workers at the same election in the same county sometimes give contradictory instructions. Clear, published guidance — available on the office’s website and provided in poll worker training — would reduce these inconsistencies and make the observation process smoother for everyone.
The office should publish a one- or two-page guidance document covering the basic external verification questions. Staff at the polling places should know the office’s policy and apply it consistently; inconsistency between polling places creates problems. The guidance should distinguish between voting hours (when restrictions are tighter) and post-closing periods, since the rules are typically different. It should include the legal basis — referencing the relevant state statutes or office policies — so external parties can verify the basis for the rules.
Texas’s rules around poll watchers (Texas Election Code Chapter 33) are relatively clear and permissive, allowing watchers to be present during the closing procedure and (in some circumstances) to record poll tapes before sealing. Other states are less clear. AC’s recommendation: every state should publish guidance at least as clear as Texas’s, even if the substantive rules differ.
7. Use high-legibility fonts on poll tapes
Voting equipment should be configured to print poll tapes using fonts where visually similar characters are clearly distinguishable. Specifically, the digits 6 and 8 should be unambiguously different, the digit 1 and the letter I should be distinguishable, and the digit 0 and the letter O should be distinguishable.
This is a narrowly technical recommendation grounded in AC’s transcription experience. Some voting equipment uses fonts (notably variants of Proxima Nova) where 6 and 8 are nearly identical at the print resolution typical of thermal poll tapes. This causes transcription ambiguity that must be resolved through contextual analysis — a time-consuming process that introduces the possibility of error. Selecting a more legible font is a trivial configuration change that would improve the accuracy of both internal and external verification.
8. Maintain consistent naming between poll tape headers and official result labels
The contest names, choice names, and precinct identifiers printed on poll tape headers should match exactly the labels used in officially published results. When they do not match — when the poll tape says “President of the United States” but the official results say “PRESIDENT” or “US President” — the comparison process requires a manual mapping table, which introduces the possibility of matching errors.
Naming consistency is a simple operational improvement that benefits everyone who works with election data, not just AC. Internal quality control at the elections office would also benefit from consistent naming across artifacts.
15.2 — Recommendations to Election System Designers and Legislators
These recommendations are addressed to the people who design voting equipment, build election management software, set state-level election policy, and write the laws that govern how elections are administered. These actors shape the environment in which elections offices operate and in which independent verification is possible.
1. Voting equipment should produce readable, standardized poll tapes
Voting machine vendors should design equipment that produces poll tapes with clear formatting, high-legibility fonts, consistent header structures, and complete information (machine ID, precinct ID, election date, contest names, choice names, vote totals, and any relevant signatures or timestamps). The tape format should be standardized within a vendor’s product line and ideally across vendors, so that a reader (human or machine) who knows how to read one tape can read any tape.
The operational details matter. Printing should be high-contrast (black on white, with sufficient ink density to remain legible after handling). Fonts should be chosen so that digits are unambiguous — specifically avoiding fonts where 6/8, 3/5, or 0/O can be confused. Header information (polling place identifier, precinct, voting method, election date, machine identifier) should be at the top of the tape and clearly visible. Contests and choices should be in a consistent column layout that doesn’t shift across tapes from the same machine type. The paper should be sturdy enough to remain legible when posted on a wall or door for an extended period. And the human-readable values should always be present and complete — some voting equipment uses QR codes or other machine-readable encodings that, while useful for some purposes, should supplement the human-readable values, not replace them.
2. Equipment should support straightforward data export
Voting equipment and election management systems should support straightforward export of results data in standard formats (CSV, JSON, XML). CSV is the lingua franca of data interchange; voting equipment should support CSV export at minimum. The export should be available to elections office staff without requiring vendor assistance, specialized software, or proprietary tools. Data export should be a routine administrative function, not a special technical operation. The structure of exported data should be documented in detail, so that anyone can write tools to process it. Export should not require running the voting equipment vendor’s proprietary software or paying licensing fees. And export should be available at the precinct × voting method × contest × choice level, not just at higher aggregations.
3. Election Management Systems should maintain transparent audit trails
The election management systems that aggregate precinct-level results into county-level and state-level totals should maintain a transparent, immutable audit trail showing every step of the aggregation. Trust in the reporting layer depends on the ability to trace any reported number back to its source. An audit trail that records every change makes the system trustworthy in a way that an opaque black box cannot be.
Any modification to a vote total — whether from importing a new poll tape, correcting an entry error, applying a rule change, or anything else — should be logged with a timestamp, the user who made the change, and the reason for the change. When a value is changed, the old value should be preserved in the audit trail, not overwritten. The audit trail itself should be in a format that can be reviewed by external auditors, with appropriate protections for staff privacy. And it should not be possible for any user (including system administrators) to change vote totals without leaving a trace.
4. State agencies should standardize results data formats
State election agencies should establish standardized data formats for the publication of official results. The standard should specify the granularity (precinct × voting method × contest × choice), the field names, the file format (CSV preferred), and the publication timeline. Results should be published promptly — available in the state-level format within a few days of the county-level results being available. Standardization across counties within a state would dramatically reduce the effort required for both internal quality control and external verification.
Currently, the format and quality of official results varies enormously by county within the same state. This variation makes statewide analysis much harder than it should be — anyone wanting to compare results across counties has to deal with each county’s specific format. State-level standardization would reduce this barrier substantially.
North Carolina’s State Board of Elections publishes statewide bulk results data that AC has used as the canonical source for NC analyses. This is the model AC recommends to other states.
5. Federal standards should support trustworthy reporting
Federal election standards (such as those promulgated by the Election Assistance Commission) should include specific requirements for vote reporting transparency, not just vote counting accuracy. The standards should require that voting equipment supports the kinds of verification described in this manual. Standards should address: poll tape formatting and legibility, results data publication formats, audit trail requirements for aggregation systems, and support for external verification. Where possible, the federal standards should reference open data formats (CSV, standard JSON schemas) rather than vendor-specific formats. And the standards should be updated regularly to reflect lessons learned from real elections, including from independent verification efforts like AC’s.
6. Require independent verification of vote reporting as a matter of law
The most consequential recommendation in this section: state and federal legislators should require independent verification of vote reporting, analogous to the independent financial audits required of publicly traded companies by the SEC.
The SEC analogy (developed in Section 13.3) is direct. Publicly traded companies are required to submit to independent financial audits because their financial statements determine how billions of dollars in capital are allocated. Elections determine who governs — decisions with even greater consequences. Yet elections have no equivalent requirement for independent verification. The entities that produce the results (elections offices) are also the primary entities responsible for verifying those results. This is the equivalent of allowing companies to audit their own financial statements — a practice that the financial world abandoned a century ago because it was unreliable.
AC is not proposing that it should be the mandated auditor. AC is proposing that independent verification should be required, that the methodology should meet rigorous standards (comparable to Generally Accepted Auditing Standards in financial auditing), and that the verifying entity should be genuinely independent of the entity producing the results. How states implement this requirement — whether through non-profit organizations, academic institutions, commercial firms, or some other mechanism — is a matter for legislative design.
15.3 — Recommendations to Voters and the Civic Ecosystem
These recommendations are addressed to individual citizens, civic organizations, advocacy groups, journalists, donors, and anyone else who cares about the trustworthiness of elections. AC’s recommendations to this audience are simpler and more personal than the institutional recommendations above.
1. Use Actual Vote yourself
The single most direct thing any citizen can do to strengthen vote reporting verification is to participate. If you live in a jurisdiction where Actual Vote can work, consider participating yourself. Download the app, practice with the practice poll tape, go to a polling place on election night, and record the posted tapes. One person at one polling place adds one precinct to the verification record. A hundred people at a hundred polling places adds a hundred. The system scales with participation. There is no minimum scale of participation — recording a single poll tape from your own neighborhood polling place is a useful contribution.
The steps are straightforward: read this manual (or at least the parts relevant to your situation — Section 7 for access methods, Section 10 for operational practice, Section 10.6 for safety); download the app from the App Store or Google Play; make a practice recording using the practice poll tape PDF before going to a real polling place; plan your visit ahead of time using the guidance in Section 7 and Section 10.2; coordinate with AC if you’re going to be doing significant work, by emailing Jason Flatley; and be patient with the system — it’s small and volunteer-driven, and your submission will be processed but not always immediately.
2. Support independent verification organizations
If you support the goal of trustworthy elections but cannot participate directly, consider supporting organizations doing independent verification work. AC is one such organization; there are others (Section 13.5 maps the landscape).
Independent verification work is chronically under-funded relative to its importance. Most election integrity funding goes to advocacy organizations, partisan operations, or elections offices themselves. Funding for independent non-partisan verification is a small part of the total. Marginal donations to verification organizations have outsized impact. Citizens who cannot personally participate in recording can support the work financially. Donations to AC (a 501(c)(3) non-profit; donations are tax-deductible) fund analysis operations, app development, volunteer coordination, and the expansion of AV’s reach to new jurisdictions. As described in Section 10.5, remote volunteer roles exist for people who can’t record locally but can contribute in other ways. If you know people who care about election integrity and might support AC’s work, introductions are valuable. And if you have a public platform — a newsletter, a podcast, a social media following — consider amplifying AC’s findings to your audience.
3. Educate yourself about how your local elections work
Most voters know very little about the mechanics of their local elections: how ballots are counted, how results are aggregated and reported, who administers the process, and what oversight exists. Understanding these mechanics is the foundation for being able to evaluate claims about election integrity — from any direction.
The public discourse about election integrity is full of confident claims based on shallow understanding. People assert things about voting machines, vote reporting, and election fraud without knowing the details. This shallow discourse drives the polarization that makes serious election integrity work harder. A more educated public — even if the education extends only to a small minority — would shift the discourse toward more substantive engagement. People who understand the difference between an apparent discrepancy and a true discrepancy are less easily manipulated by claims that conflate the two. People who understand the matched poll tape caveat are less easily manipulated by claims that AC’s findings prove or disprove things they don’t actually address.
AC recommends that citizens take the time to learn. Read this manual — the technical sections (Section 4 for system architecture, Section 7 for access methods, Section 8 for methodology) are written to be understandable by motivated non-experts. Read the canonical AC documents: the Why You Can Trust Actual Vote essay, the About America Counts page, and the published analysis reports (NC 2024 primary, 2024 General Election News Page) are good starting points. Read about how your local elections office works — most county elections offices have informational content on their websites about how voting and tabulation work in their jurisdiction. This local knowledge is valuable. And read about the broader landscape: risk-limiting audits, election observer programs, voting equipment certification processes — there’s a lot to learn beyond AC’s specific niche.
4. Demand trustworthy, transparent reporting from your elections office
Citizens should expect and demand that their elections offices publish results in accessible formats, follow established posting procedures, respond to legitimate inquiries about vote reporting, and cooperate with independent verification efforts. These are reasonable expectations of any public institution. Elections offices respond to the constituencies they serve. Voters who are paying attention and asking for specific things have influence. Voters who aren’t paying attention have little influence.
The ways to engage are concrete. Write to your county elections office — a polite letter or email asking about the office’s practices and (where relevant) suggesting specific improvements is a small but meaningful contribution. Attend public meetings — many elections offices have advisory boards or commissions that meet publicly, and attending and speaking up is a way to be heard. Vote in elections for elected officials who oversee elections — in many jurisdictions, the people who run elections are themselves elected (Secretaries of State, county clerks, supervisors of elections), and your vote in those elections is your direct influence over election administration. And don’t be hostile. As discussed in Section 8.8 and Section 15.1, the right tone for engagement with elections offices is cooperative, not adversarial. Hostility from the public makes elections officials defensive; cooperative engagement makes them open.
5. Hold both political camps to the same standard
Election integrity is not and should not be a partisan issue. When evaluating claims about election integrity, apply the same standard regardless of which political camp is making the claim. A claim of fraud should be evaluated on the evidence, not on the party affiliation of the claimant. A defense of election integrity should likewise be evaluated on the evidence.
The polarization of election integrity discourse partly results from people applying different standards to claims from different camps. Claims from one’s own side are believed easily; claims from the other side are dismissed easily. This asymmetry is the opposite of how careful thinking works. A symmetric standard — believing things based on evidence, not based on tribal affiliation — is harder to apply but produces better conclusions. AC’s non-partisan stance (Section 13.2) is built on this principle, and AC’s findings are designed to be useful to people who want to apply the symmetric standard. Citizens who adopt this even-handed stance strengthen the broader credibility of the election integrity project and reduce the risk that legitimate concerns are dismissed as partisan gamesmanship.
The operational guidance: notice when you’re applying asymmetric standards — when you believe a claim from one camp easily and dismiss a similar claim from the other camp, ask yourself whether the evidence is actually different or whether your reaction is based on the source. Look for primary sources rather than relying on partisan summaries of evidence. Be willing to be wrong — if the evidence in a specific case turns out not to support what you initially believed, update your belief; this is uncomfortable but necessary for good thinking. And hold yourself to the standard you’d want others to hold: if you would expect a member of the other political camp to update on contrary evidence, you should expect the same of yourself.
15.4 — Data Science and Measurement
This subsection is new to this edition of the manual. It addresses a question that donors, partners, journalists, and AC itself increasingly need to answer: how do we measure the impact of vote reporting verification?
AC already reports the basic outputs of every Comparison Analysis: the number of poll tapes transcribed, the number of individual values compared, the number of discrepancies found, and the disposition of those discrepancies (resolved, unresolved, referred to the elections office). These are the mechanical facts of the analysis. They tell the reader what AC did. They do not, by themselves, tell the reader what that work was worth.
The challenge of impact measurement is that the most important effects — deterrence, confidence-building, and systemic quality improvement — are inherently difficult to measure. An error that was never committed because the perpetrator knew AV was watching does not show up in any dataset. Increased public confidence in election results is real but hard to attribute to any single cause. Procedural improvements at elections offices may be influenced by AV’s presence but are rarely attributable to it alone.
This subsection describes the metrics AC has adopted for quantifying impact, the metrics it has considered and set aside, and the data science methods it uses to target and interpret its verification work.
Margin proximity
The most important impact metric AC has identified is what it calls margin proximity: the relationship between the discrepancies found in an analysis and the margin of victory in the affected race.
The intuition is simple. A discrepancy is a difference between a value on a poll tape and the corresponding value in the official reported results. Discrepancies vary in magnitude. A transposition error that swaps two digits in a single precinct’s count might affect a handful of votes. An aggregation error that drops an entire precinct’s results from a county total might affect thousands of votes. The raw count of discrepancies does not distinguish between these cases, and neither does the total number of comparisons performed.
What distinguishes them — what determines whether a discrepancy matters for the outcome of an election — is how the magnitude of the discrepancy compares to the margin of victory in the relevant race. This is margin proximity.
Definition. For a given race in a given jurisdiction, the margin proximity ratio is:
aggregate discrepancy magnitude ÷ winning margin
where aggregate discrepancy magnitude is the sum of the absolute values of all discrepancies found in that race, and the winning margin is the difference in votes between the first-place and second-place finishers.
Example. Suppose AC analyzes vote reporting in Wake County, NC for a state legislative race. AC’s Comparison Analysis compares poll tape values against official results for every precinct in the county and finds three discrepancies:
- Precinct 04-07: the official results report 312 votes for Candidate A, but the poll tape shows 321. Magnitude: 9 votes.
- Precinct 11-02: the official results report 189 votes for Candidate B, but the poll tape shows 198. Magnitude: 9 votes.
- Precinct 22-01: the official results report 1,407 votes for Candidate A, but the poll tape shows 1,047. Magnitude: 360 votes.
The aggregate discrepancy magnitude is 9 + 9 + 360 = 378 votes. (Note that the direction of the errors matters for investigating whether the outcome might have changed, but for the margin proximity ratio we use absolute values because we are measuring the total magnitude of reporting uncertainty, not the net partisan effect.)
Suppose the official results show Candidate A winning by 5,200 votes. The margin proximity ratio is 378 / 5,200 = 0.073, or about 7.3%. This means the total magnitude of detected reporting errors amounts to about 7% of the winning margin. That is notable — worth investigating and correcting — but the errors are not large enough, even in the worst case, to have changed the outcome.
Now suppose instead that the same race was decided by 400 votes. The margin proximity ratio becomes 378 / 400 = 0.945, or about 94.5%. The discrepancies are nearly as large as the margin itself. In this case, the outcome literally depends on whether the reporting was done correctly — and the analysis has surfaced evidence that it was not. This is a qualitatively different situation from the 7% case, even though the discrepancies are identical.
Why margin proximity is the right metric. Margin proximity connects the mechanical work of the analysis (finding discrepancies) to the thing that every audience — lawyers, politicians, voters, donors — actually cares about: whether the right person won. It does not require any modeling assumptions, any priors, or any subjective judgments beyond the data AC already collects. It is computed per-race, which is the natural unit of analysis for election outcomes.
Margin proximity also provides a natural scale for characterizing the severity of reporting problems. A margin proximity ratio below 0.01 (discrepancies less than 1% of the margin) is a clean bill of health: the reporting was verified and the errors found are non-contest-flipping. A ratio between 0.01 and 0.10 means the errors are detectable but small relative to the margin. A ratio above 0.10 means the errors are significant enough to warrant investigation. A ratio above 0.50 means the reporting errors are large enough that the outcome is in genuine question. And a ratio above 1.0 means the detected errors are larger than the margin — the reported outcome cannot be trusted without resolution of the discrepancies.
These thresholds are guidelines, not rigid cutoffs, and AC does not claim that any specific threshold has a special statistical significance. But they provide a vocabulary for communicating the stakes of an analysis in terms that are immediately understandable.
When the analysis finds no discrepancies. If AC’s analysis finds zero discrepancies — as it did in the NC 2024 Primary across 321,502 votes and seven counties — the margin proximity ratio is zero for every race. This is a strong positive result. It means AC independently verified the vote reporting and found it to be accurate, to the resolution of its methodology, in every race in the analysis. The margin proximity metric communicates this clearly: a ratio of zero means the analysis found no reporting errors that could have affected any outcome.
Limitations. Margin proximity measures the impact of what AC found. It does not measure the impact of what AC didn’t find. An analysis might miss discrepancies because AC’s methodology has finite coverage (not every precinct may have a matched poll tape) or because certain kinds of errors are invisible to the methodology (Section 8’s matched poll tape caveat). Margin proximity should therefore be interpreted as a lower bound on reporting uncertainty, not as a complete accounting.
Coverage ratio
A simpler but complementary metric is the coverage ratio: the fraction of total votes cast in a jurisdiction that were included in AC’s analysis.
votes analyzed ÷ total votes cast
If a county cast 200,000 votes and AC’s analysis included precincts accounting for 150,000 of those votes, the coverage ratio is 0.75, or 75%.
Coverage ratio serves a different purpose from margin proximity. Where margin proximity measures the severity of what was found, coverage ratio measures the breadth of the verification effort. A high coverage ratio means AC has verified a large fraction of the jurisdiction’s vote reporting. A low coverage ratio means there are large gaps — precincts where the reporting has not been independently checked.
Coverage ratio is particularly useful as a motivational and planning metric. It maps naturally to a visual representation: a map of a county or state where verified precincts are shaded and unverified precincts are not. This kind of visual makes the case for broader participation more effectively than any number. It answers the question “where do we still need to go?” in a way that volunteers and donors immediately understand.
It is also useful for characterizing the strength of an analysis. An analysis that achieves a coverage ratio of 0.95 and finds no discrepancies is a much stronger result than an analysis that achieves a coverage ratio of 0.30 and finds no discrepancies. The margin proximity ratio may be zero in both cases, but the second analysis has verified less than a third of the jurisdiction’s reporting.
Metrics that seem appealing but are not (yet) useful
Several other impact metrics have been considered and, at least for now, set aside. They are documented here because they are natural ideas that come up in discussion, and it is worth being explicit about why AC has not adopted them.
Election Integrity Points and composite scores. The idea of a single composite score — “this analysis earned 107 Election Integrity Points” — is appealing for communication purposes. A single number is easier to put in a headline, easier to compare across analyses, and easier to track over time. But composite scores require weighting the components, and the weights are inherently subjective. How much is a 0.05 margin proximity ratio worth relative to a 0.80 coverage ratio? There is no empirical basis for choosing weights, and any choice can be challenged. A composite score that obscures its inputs is less trustworthy than the inputs themselves, and AC’s audience — lawyers, political scientists, sophisticated donors — is exactly the audience that will (rightly) interrogate the weighting. AC reports its component metrics separately and lets the reader draw their own conclusions.
Bayesian prior-to-posterior confidence. The idea here is to model the “confidence gained” by an AV analysis as a Bayesian update. Before the analysis, the public has some prior probability that vote reporting in a jurisdiction was done correctly. After the analysis, the posterior probability is higher (if no discrepancies were found) or lower (if discrepancies were found). The ratio of posterior to prior represents the “value” of the analysis. The concept is sound in principle. In practice, it requires specifying the prior — the probability that vote reporting was correct before AC looked. This is the step where the approach breaks down. There is no consensus on the prior. Some people assume reporting is essentially always correct; their prior is very close to 1, and the update from an AV analysis is small. Others believe reporting is routinely compromised; their prior is much lower, and the same analysis produces a large update. The metric becomes an argument about priors rather than a measurement of impact. AC’s view is that the data from the analysis speaks for itself and does not need to be filtered through a prior that AC would have to defend.
Difference-in-differences and causal inference. A more ambitious approach would treat AV’s presence as a “treatment” and attempt to measure its causal effect on some outcome — voter confidence, election integrity perceptions, or a Social Well-Being index. The standard econometric approach (difference-in-differences) would compare outcomes in jurisdictions where AC performed analyses to outcomes in similar jurisdictions where it did not, controlling for socioeconomic and demographic factors. This approach requires three things AC does not currently have: a well-defined dependent variable that is measurable at the jurisdiction level, a sufficient number of analyzed jurisdictions to support statistical inference, and a plausible identification strategy that addresses the fact that AC’s choice of where to analyze is not random. AC tends to analyze jurisdictions where there is volunteer interest or where prior evidence suggests vulnerabilities — precisely the jurisdictions that are likely to differ from the average on the outcome variables of interest. Without addressing this selection bias, any estimated “effect of AV” would conflate the impact of the analysis with the pre-existing characteristics of the places AC chooses to analyze. This is not a permanent objection. As AC’s coverage grows and the number of analyzed jurisdictions increases, causal inference approaches may become viable. But they are not viable today, and AC prefers to report metrics it can defend rather than metrics it cannot.
Seeded-error proficiency testing
A verification pipeline should itself be verified. AC’s recommendation — to itself, and to any organization doing comparable work — is seeded-error proficiency testing: deliberately inject known synthetic errors into the transcription and comparison pipeline under blind conditions (the transcribers and analysts processing the seeded material do not know which items are seeded), and measure empirically what fraction of the injected errors the pipeline detects and what residual error rate survives it.
This is the standard practice of forensic laboratories, where blind proficiency tests are the accepted way to establish that a detection process actually detects, at a measured rate, rather than being assumed to. The same logic applies to a poll-tape comparison pipeline: the claim “our investigation process catches transcription and comparison errors” is an empirical claim, and seeded-error testing is the experiment that tests it. The design of the test (what kinds of errors are seeded, at what rates, at which pipeline stages) and the measured results — detection rates and residual error rates — should both be published, so that readers of AV reports can calibrate how much confidence the pipeline’s clean results deserve.
Urgency-targeting
Not all jurisdictions and elections are equally important targets for AV operations. AC is developing pre-election targeting methods that identify the highest-priority jurisdictions based on multiple factors: historical patterns of vote reporting problems, known vulnerabilities in the jurisdiction’s voting equipment or reporting software, the expected closeness of races (close races mean that smaller errors can change outcomes), demographic factors correlated with historical under-reporting or suppression, and the jurisdiction’s track record of cooperation with or resistance to external verification.
The targeting framework combines quantitative inputs (election history data, polling data, equipment databases, demographic data) with qualitative inputs (AC’s institutional knowledge about specific jurisdictions, partner organization intelligence, media reporting on election administration issues). The goal is not to predict where fraud will occur — AC does not make such predictions — but to allocate limited volunteer and analysis resources where they will produce the most valuable verification.
Inferential statistics from AV results
A recurring question in AC’s work is what can be inferred about the unaudited portions of an election from the audited portions. If AC analyzes 52 precincts in a state with 2,000 precincts and finds zero discrepancies, what does this tell us about the other 1,948?
The honest answer is: less than one might hope. Classical statistical inference (confidence intervals on population proportions based on random samples) requires assumptions — random sampling, independence of observations, common error rates across precincts — that are difficult to justify in this context. Vote reporting errors are not randomly distributed; they depend on the specific equipment, software, personnel, and procedures at each location. A jurisdiction with a centralized software bug might have errors at every precinct; a jurisdiction with one careless poll worker might have errors at only one precinct. Without knowing the structure of the error distribution in advance, inferring from a sample to a population is unreliable.
Bayesian approaches offer more nuance. If one starts with a prior belief about the probability of vote reporting error (informed by historical data, equipment characteristics, and other factors) and updates that prior with AV’s observations, the posterior probability can be meaningfully lower than the prior. But the result depends heavily on the choice of prior, and AC’s operational data is still too small to establish strong empirical priors.
AC is honest about these limitations. AV’s findings are strongest when interpreted as evidence about the specific precincts analyzed, not as statistical inferences about the broader population. The rising-tide theory of change (Section 13.5) addresses the limitations of inference by targeting coverage growth: as more precincts are covered, the inferential gap shrinks.