OCR evaluation against source references

70 pages · four execution profiles · 280 page/profile slots

Open the four-model interactive demo ↗

Inspect source pages + evaluation references ↗

Development evaluation: references were created by the assistant from source pages. The user inspected examples and endorsed the method; these are not independently human-annotated labels. Unresolved handwriting is excluded. Raw OCR only: Gemma and pipeline repairs receive no credit.

Overall interpretation for this corpus
  1. dots.mocr BF16 — strongest balance; 91.67% visual recall, 80.96% row/value match.
  2. dots.mocr Q4/Q8 — lowest scoped text error, 0.67%; smaller-GPU production choice.
  3. PaddleOCR-VL-1.6 — best row/value match, 86.58%, and heading detection.
  4. Unlimited-OCR 2 — 55.62% row/value match and two missing handwriting outputs.

Qualitative judgment across metrics, not a combined score. The tested Q4/Q8 profile runs on the 6 GB GPU; BF16 reached 8.53 GiB peak PyTorch allocation on Colab T4, which is not total device memory or a minimum VRAM requirement.

Metrics JSONPage metricsEvaluation methodology

↑ higher is better; ↓ lower is better. — means there is no applicable reference denominator. Overall rates pool their underlying units; per-page macro averages are retained in metrics.json. No single combined winner score is calculated.

Scope, matching rules and limitations

Reference SHA-256:

Scorer SHA-256:

Excluded annotations and uncertainty

Open the full exclusion ledger

Page-by-page results

Reference links use selected-PDF page numbers. CER is scoped source-region content recovery; native output presence and usable layout blocks are shown separately. Unlimited's two historical handwriting failures remain in the denominator.