Development evaluation: references were created by the assistant from source pages. The user inspected examples and endorsed the method; these are not independently human-annotated labels. Unresolved handwriting is excluded. Raw OCR only: Gemma and pipeline repairs receive no credit.
Overall interpretation for this corpus- dots.mocr BF16 — strongest balance; 91.67% visual recall, 80.96% row/value match.
- dots.mocr Q4/Q8 — lowest scoped text error, 0.67%; smaller-GPU production choice.
- PaddleOCR-VL-1.6 — best row/value match, 86.58%, and heading detection.
- Unlimited-OCR 2 — 55.62% row/value match and two missing handwriting outputs.
Qualitative judgment across metrics, not a combined score. The tested Q4/Q8 profile runs on the 6 GB GPU; BF16 reached 8.53 GiB peak PyTorch allocation on Colab T4, which is not total device memory or a minimum VRAM requirement.
↑ higher is better; ↓ lower is better. — means there is no applicable reference denominator. Overall rates pool their underlying units; per-page macro averages are retained in metrics.json. No single combined winner score is calculated.
Scope, matching rules and limitations
Reference SHA-256:
Scorer SHA-256:
Excluded annotations and uncertainty
Open the full exclusion ledger
Page-by-page results
Reference links use selected-PDF page numbers. CER is scoped source-region content recovery; native output presence and usable layout blocks are shown separately. Unlimited's two historical handwriting failures remain in the denominator.