How AI receipt scanning works, end to end
A receipt scan passes through five stages: capture, image cleanup, text recognition, field extraction, and categorisation. Each stage hands the next one a slightly different kind of data — pixels, then characters, then labelled values, then an accounting entry — and each stage can only work with what the one before it produced. That chain is why a bad photo cannot be rescued by a clever model later on.
Most explanations of receipt scanning stop at “AI reads it”. That’s not wrong, but it hides the part that matters when something goes wrong, which is which stage failed. Here is the whole path.
Stage 1: capture
The camera. This is the only stage you control directly and the only one where a mistake is unrecoverable — everything downstream is inference over the pixels you gave it. Detail that isn’t in the photo cannot be recovered by processing.
What capture is trying to produce: the whole slip, in frame, roughly flat, evenly lit, in focus, at enough resolution that the smallest printed characters are several pixels tall. That last one is the constraint people underestimate. Receipt totals are printed large; the date and the card’s last four digits often are not.
Stage 2: cleanup and rectification
Before anything tries to read text, the image gets normalised. Typical steps: find the receipt’s edges against the background, correct the perspective so the slip becomes a rectangle rather than a trapezoid, straighten any rotation, convert to greyscale, and push contrast so faint thermal printing separates from the paper.
Curl is handled here too, and handled imperfectly. A receipt that was rolled up has a curved baseline; software can flatten mild curvature but a tight curl distorts characters in ways that no amount of contrast fixes. This is why “press it flat” is genuinely the highest-value habit in the whole process.
Stage 3: text recognition
OCR turns pixels into characters. Modern recognition is very good at clean printed text and much less good at the specific things receipts are full of: dot-matrix output, faded thermal ink, condensed fonts, and columns whose alignment is done with spaces rather than any structural markup.
The output at this stage is not “a receipt”. It is a pile of text fragments, each with a position on the page and usually a confidence value. Nothing yet knows which fragment is the total.
Stage 4: field extraction
This is the part people mean by “AI”, and it is a genuinely hard problem, because a receipt has no schema. There is no rule that the total appears at the bottom, no rule about what it’s labelled — TOTAL, AMOUNT DUE, BALANCE, or nothing at all — and no rule that the largest number on the slip is the one you want. Tax lines, subtotals, tips, cash tendered and change are all numbers in the same neighbourhood as the total.
Extraction works by combining several kinds of signal:
Position. Vendor name near the top, totals near the bottom, line items in the middle band. Weak individually, useful in aggregate.
Nearby words. A number to the right of the word “total” is a strong candidate; a number under “change” is a strong non-candidate.
Internal consistency. Line items should sum to a subtotal, subtotal plus tax should equal the total. Arithmetic that agrees is evidence the fields were labelled correctly. Arithmetic that doesn’t agree is the single most useful error signal available.
Format. Dates look like dates, currency looks like currency, and a card number fragment is four digits in a very particular context.
The output is a set of typed fields with confidences attached: vendor, date, currency, subtotal, tax, total, sometimes line items, sometimes payment method.
Stage 5: categorisation and posting
Now the extracted vendor and amount get mapped onto your chart of accounts — office supplies, travel, meals, software. This stage knows things the receipt doesn’t: your past decisions about this vendor, which account you used last time, what the rest of your business spends on.
Two mechanisms coexist. Rules are explicit and deterministic (“this vendor always posts to Software”), and they should always win when they exist, because they encode a decision you already made. Learned categorisation handles everything else by generalising from your history, which is why it works poorly on day one and well after a few hundred receipts.
Where the chain actually breaks
In our experience the failures cluster, and they cluster early:
Stage 1 dominates. Cropped edges, glare across the total, motion blur, and a photo taken at such an angle that the bottom of the slip is a third the width of the top. Nothing downstream recovers these.
Stage 3 fails on faded paper. A thermal receipt that has spent a summer in a car is not a scanning problem, it’s a physics problem, and the answer is capturing it sooner rather than processing it harder.
Stage 4 fails on ambiguity, not illegibility. The text was read perfectly and the wrong number was chosen — a tip mistaken for a total, a subtotal mistaken for a total, a card’s last four digits mistaken for a date. These are the dangerous errors because the result looks entirely plausible.
Stage 5 fails silently on new vendors. A first-time vendor gets a guess, and a guess that lands in the wrong account is invisible until someone reviews the account.
Why this is worth knowing
Because the fix differs per stage. If your scans are wrong, “try a different app” is only the right response to a stage 3 or 4 problem. A stage 1 problem follows you to every app you try, and a stage 5 problem is solved by writing down a rule rather than by changing software.
The practical version: when a receipt comes out wrong, look at the image first. If the number you wanted is clearly legible to you in the photo, the software has a real bug worth reporting. If it isn’t, no software was ever going to get it.