OCR Accuracy Benchmark — A Methodology for Bank Statements

If you only take one idea away, make it this. A statement carries its own answer key. The bank prints a running balance, so opening plus every transaction must equal the closing figure. That arithmetic lets you score completeness objectively, per statement, without anyone typing out a "correct" version by hand.

The four levels of OCR accuracy

People ask "how accurate is the OCR?" as though there were a single figure. There isn't. For bank statements, accuracy lives at four levels, and each answers a different question. A benchmark that quotes one without naming the level is hiding more than it shows.

Character-level accuracy is the rawest measure: of every character the engine read, how many match the source? It's the classic OCR metric, often expressed as character error rate, and it's useful for spotting which fonts or scan qualities trip the engine. But on its own it flatters. A statement can hit very high character accuracy and still be unusable, because one wrong digit in an amount column matters far more than ten correct characters in a merchant name.

Field-level accuracy asks whether each cell — a date, a description, an amount — came across whole and correct. This is closer to what you actually use. A single transposed digit fails the whole amount field even if every other character in it was right.

Transaction-level accuracy raises the bar again: a transaction passes only if its date, description and amount are all correct together. This is the bookkeeper's unit, because a transaction is what you post. One misread field sinks the row.

Balance-level accuracy is a yes/no question per statement: does the extracted data reconcile? Opening balance, plus the sum of every credit and debit, should land exactly on the printed closing balance. This is the level that catches the error the other three can miss entirely — a transaction dropped wholesale. A missing row leaves no wrong character and no wrong field to count. The cells that remain are all correct. Only the maths reveals that a line went absent.

Report all four and the picture is honest. Character accuracy shows raw reading quality. Field and transaction accuracy show usable output. Balance accuracy shows completeness — the thing almost everyone assumes and almost nobody verifies.

Building a fair test set

Most weak benchmarks fail before a single character is read, at the test-set stage. The set is too small, too clean, or too vague to mean anything. A fair OCR benchmark for bank statements is built deliberately across the dimensions that actually move accuracy.

The biggest split is document origin. A text-based PDF downloaded from online banking stores its characters as data, so the tool reads them directly and OCR barely does any work — accuracy there is near-exact. A scanned or photographed statement is just an image, and OCR has to infer each character from pixels. Blend those two into one number and you bury the scanned-document weakness under the easy wins. Keep them in separate buckets, always.

Within the scanned bucket, vary the conditions that degrade an image in real life: resolution (a crisp 300 dpi scan versus a low-res phone photo), skew and rotation, contrast and fading (old or thermal-printed statements lose stroke detail), and physical marks like fold lines, staple holes or highlighter. A test set made only of clean 300 dpi scans tells you nothing about the messy file a client will actually send.

Layout matters as much as image quality. Banks format statements very differently, and the format predicts the error better than the engine does. So stratify by layout too: single debit/credit columns versus a combined column with a sign, descriptions that wrap onto a second line, statements where the date prints only on the first transaction of each day, and multi-page files where rows can be lost or duplicated at the page break.

State the size and composition of the set plainly: how many statements, which banks, which countries, and the mix across those rows. A described sample is what makes a result credible; "99% accurate" with no sample behind it is not.

Reconciliation as built-in ground truth

Here's the methodological wedge, and it's a real one. Scoring OCR normally needs a reference — a human-keyed "true" version of each statement to compare the output against. That's slow, subjective at the edges, and it doesn't scale past a handful of files.

The running balance sidesteps that for completeness and amount integrity. Banks print balances: at least an opening and closing figure, often a balance after each line. That sequence is the bank's own arithmetic, shipped inside the document. Rebuild it from the extracted transactions. If the maths closes, no transaction was lost and the amounts sum correctly. If it doesn't, you have proof, not a guess, that the extraction is incomplete or wrong, with nobody hand-keying a reference copy.

Export Bank Statement runs exactly this check on every conversion. It reconstructs the running balance from the extracted lines and compares it against the figures the bank printed. When they agree, every transaction is accounted for. When they don't, the statement is flagged so you look before you trust it. For a benchmark, that turns balance-level accuracy into a metric you can compute automatically across the whole test set — a reconciliation pass rate.

Two honest limits, because pretending the check is infallible would undercut the point:

• Reconciliation proves completeness and amount integrity, not description quality. A merchant read as "AMZN Mktp" instead of "Amazon" still reconciles, because the maths only sees amounts. Description accuracy needs its own scoring pass against a reference.
• Two errors can cancel. A line read 10 too high and another read 10 too low still balances. It's rare, rarer still on a long statement, but a methodical benchmark names it rather than ignoring it.

So reconciliation isn't the whole benchmark. It's the strongest single objective signal, it scales, and it catches the most dangerous failure — silent data loss.

What moves OCR accuracy

Accuracy isn't a fixed property of an OCR engine. It's a property of the engine and the statement together — which is why the test set above carries so much weight. These are the factors that shift the result, roughly in order of impact.

Image quality on scans. This dominates everything else. Resolution, lighting, skew and contrast decide how clean the pixels are before recognition even starts. We've watched the same statement go from spotless to riddled with misreads purely because it was photographed in bad light. Text PDFs skip all of this, which is why the origin split is the first cut in any benchmark.

Character confusability. When OCR slips, the mistakes aren't random — they cluster on shapes that look alike. A 0 read as an 8, a 1 as a 7, a 5 as a 6, a comma swallowed into a thousands separator. The painful one is a single wrong digit in an amount. The spreadsheet still looks tidy; the figure is just quietly wrong until a balance won't tie.

Layout and table structure. Recognising characters is only half the job; the tool also has to keep them in the right column. Skew, combined debit/credit columns and wrapped descriptions all give the table-detection step a chance to misalign a row. For how that stage actually works, see how OCR works on bank statements.

Number formatting. Decimal commas versus points, currency symbols inside the cell, trailing "CR"/"DR" markers — any of these can be misparsed, turning 1,234.56 into 1.23 or flipping a sign. These surface as amount errors, the exact failures the reconciliation check is built to catch.

For measuring conversion quality across all document types, the extraction accuracy benchmark covers field and transaction scoring beyond OCR alone.

Where the tool fits — and where it stops

Export Bank Statement converts bank-statement PDFs to clean Excel and CSV, reads scanned and photographed statements via OCR, and runs the running-balance reconciliation on every file so you know whether the output is complete. It also exports in the native bank-import CSV format for Xero, QuickBooks and Zoho Books.

The honest boundary: it does not push transactions into your accounting software through a live bank-feed API — that route needs partner certification we don't claim. The workflow is convert, confirm it reconciles, then import the CSV. You import data you've already seen verified, rather than trusting an opaque sync.

CTA: Benchmark it yourself — convert your hardest scanned statement and check whether it reconciles. The per-file result is your own balance-level accuracy, measured on the statements you actually deal with.