OCR Pipelines

An OCR pipeline converts document images into text and structured fields. It combines classical image processing, page or text-region detection, recognition, language correction, layout reconstruction, and downstream extraction. For skewed labels and text lines, rotated object detection may be the localization step.

The OCR composition

A practical OCR system is a composition

$$I\to I^{\prime }\to R=\{r_i\}\to {\hat{t}}_i\to \hat{T}\to \hat{F},$$

where $I^{\prime }$ is the cleaned image, $r_i$ are text regions, ${\hat{t}}_i$ are recognized strings, $\hat{T}$ is reconstructed text, and $\hat{F}$ are structured fields. Recognition models often minimize CTC loss, which marginalizes over alignments between frame-level predictions and the final character sequence.

Worked example

This snippet computes edit-distance based character error rate and word error rate for an OCR hypothesis against a reference transcription.

import numpy as np
 
def edit(a, b):
    dp = np.zeros((len(a)+1, len(b)+1), int)
    dp[:,0] = np.arange(len(a)+1); dp[0,:] = np.arange(len(b)+1)
    for i, ca in enumerate(a, 1):
        for j, cb in enumerate(b, 1):
            dp[i,j] = min(dp[i-1,j]+1, dp[i,j-1]+1, dp[i-1,j-1] + (ca != cb))
    return int(dp[-1,-1])
 
truth = "INV-1042 TOTAL 39.50"
ocr = "INV-I042 T0TAL 39.5O"
print("char_edits", edit(truth, ocr), "cer", round(edit(truth, ocr) / len(truth), 3))
print("word_edits", edit(truth.split(), ocr.split()), "wer", round(edit(truth.split(), ocr.split()) / len(truth.split()), 3))

Observed output:

char_edits 3 cer 0.15
word_edits 3 wer 1.0

Three character substitutions make every token wrong under exact word matching, which is why field-level validation is often more useful than OCR text alone.

Caveats

Deskew and binarization can improve clean scans but damage faint ink. Recognition errors are not independent: one layout miss can delete an entire paragraph. In document image analysis and field extraction, entity linking may matter more than raw character error rate.

References

• Tesseract OCR documentation
• An End-to-End Trainable Neural Network for Image-based Sequence Recognition

Nav

Section — Computer Vision

← Content-Based Image Retrieval Document Image Analysis and Field Extraction →