Best OCR for Invoice Processing.

dots.ocr detected 20 layout elements with bounding boxes: Page-headers (green), Section-headers (cyan), Text blocks (green), Tables (pink), Pictures (magenta), Page-footers (purple).

Page-headers

3

Section-headers

5

Text blocks

6

Tables (HTML)

2

Pictures

1

Page-footers

3

Invoice Number

FV 1/09/2019

Net Total

12,970.00 PLN

VAT (23%)

2,983.10 PLN

Gross Total

15,953.10 PLN

Bank Account

28 1020 1127...

Advantages

• +88.6% table TEDS - Best-in-class for line items
• +Native bounding boxes - No separate detection model
• +100+ languages - Polish, German, Chinese in one model
• +Apache 2.0 - Open source, self-hostable
• +8GB VRAM - Runs on RTX 3070/4070

Limitations

• −Requires GPU for production speed
• −Formula/LaTeX recognition not best-in-class
• −May struggle with extremely dense documents
• −Tables output as HTML (need parsing)

dots.ocr via Replicate API

import replicate
import base64
import json
from PIL import Image
from io import BytesIO

PROMPT_LAYOUT_ALL = """Extract all layout elements with bounding boxes.
Output JSON array with: bbox, category, text.
Categories: Page-header, Section-header, Text, Table, Picture, Page-footer"""

def parse_invoice(image_path):
    img = Image.open(image_path).convert('RGB')
    buffer = BytesIO()
    img.save(buffer, format='JPEG')
    b64 = base64.b64encode(buffer.getvalue()).decode()

    output = replicate.run(
        "sljeff/dots.ocr:214a4fc4...",
        input={
            "image": f"data:image/jpeg;base64,{b64}",
            "prompt": PROMPT_LAYOUT_ALL,
            "temperature": 0.1,
        }
    )
    return json.loads("".join(output))

# Example output for Polish invoice:
# [
#   {"bbox": [361, 63, 513, 76], "category": "Page-header",
#    "text": "Faktura VAT nr FV 1/09/2019"},
#   {"bbox": [45, 322, 569, 382], "category": "Table",
#    "text": "<table><thead>...</thead><tbody>...</tbody></table>"}
# ]

Parse HTML Table Output

from bs4 import BeautifulSoup

def extract_line_items(cells):
    """Parse HTML table output from dots.ocr"""
    for cell in cells:
        if cell['category'] == 'Table':
            if 'Nazwa towaru' in cell['text'] or 'Description' in cell['text']:
                soup = BeautifulSoup(cell['text'], 'html.parser')
                rows = soup.find_all('tr')

                items = []
                for row in rows[1:]:  # Skip header
                    cols = row.find_all(['td', 'th'])
                    if len(cols) >= 5:
                        items.append({
                            'name': cols[1].text.strip(),
                            'quantity': cols[3].text.strip(),
                            'unit_price': cols[4].text.strip(),
                            'total': cols[-1].text.strip(),
                        })
                return items
    return []

GPT-5.4 Alternative (Direct JSON)

import base64
from openai import OpenAI

client = OpenAI()

def parse_invoice_gpt4o(image_path):
    with open(image_path, 'rb') as f:
        img_b64 = base64.b64encode(f.read()).decode()

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": [
            {"type": "text", "text": """Extract from this invoice:
- invoice_number, date, due_date
- seller (name, address, tax_id)
- buyer (name, address, tax_id)
- line_items (description, qty, unit_price, total)
- subtotal, tax_rate, tax_amount, total
Return as JSON."""},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}}
        ]}]
    )
    return json.loads(response.choices[0].message.content)