In Indian trucking, the "document management system" is usually WhatsApp. A driver fills diesel at a pump, takes a photo of the slip and sends it to the office group. Multiply that by tolls, weighbridge slips and repair bills across a fleet, and the office ends up retyping hundreds of images a month.

We built JC Fleet at JC TechLabs for exactly this workflow. One design principle shaped everything: OCR output is a draft, not a fact. This post walks through the general architecture of a WhatsApp-first OCR pipeline with a human-in-the-loop review step. The patterns apply to invoices, receipts or any messy real-world documents.

The problem with "just OCR it"

Real-world slips are hard:

  • Mixed Hindi and English text, sometimes handwritten
  • Thermal paper that has faded, photographed at an angle
  • The same slip forwarded twice by two different people
  • A vehicle number that's partly smudged and could match two trucks

If your pipeline writes the extracted values straight into an accounting ledger, you'll create confident-looking wrong numbers. Wrong numbers in a ledger are far more expensive than slow numbers.

Architecture at a glance 

WhatsApp Business webhook / direct upload │ ▼ 1. Intake & sender check ──► store original file + metadata │ ▼ 2. OCR + field extraction (hi/en) │ ▼ 3. Validation & matching (vehicle, trip, date, duplicates) │ ┌──────┴────────┐ ▼ ▼ 4a. Auto-post 4b. Review queue ──► human corrects / retries extraction (all checks pass) │ └──────┬─────────────┘ ▼ 5. Operational / accounting record (original stays linked)

1. Intake: keep the original, always

Whether a file arrives through the WhatsApp Business API webhook or a direct upload, store the original attachment first, with sender, timestamp and channel. Everything downstream references it. The reviewer needs to see the real slip, not just your parsed JSON.

Also check who sent it. Is this number mapped to a known driver or staff member? An unknown sender shouldn't be able to create expense entries automatically.

from dataclasses import dataclass, field from datetime import datetime @dataclass class IncomingDocument: doc_id: str # stable ID, e.g. WhatsApp media ID or a content hash  sender: str received_at: datetime source: str # "whatsapp" | "upload"  file_path: str content_hash: str flags: list[str] = field(default_factory=list)

2. OCR and field extraction

Run OCR with support for both Devanagari and Latin scripts, then extract the fields you care about: document type, vendor, date, vehicle reference, quantity, rate and total. Keep a per-field confidence score if your engine provides one.

The key output is not a value. It's a value plus how sure you are:

@dataclass class Extracted: value: str | float | None confidence: float # 0.0 – 1.0

3. Validation and matching

This is where most of the reliability comes from. Some checks are cheap and very effective:

Arithmetic consistency. On a fuel slip, quantity × rate should equal the total. 45 L × ₹90 = ₹4,050. If it doesn't add up, either OCR misread something or the slip itself is odd. Either way, a human should look.

def arithmetic_ok(qty: float, rate: float, total: float, tol: float = 1.0) -> bool: return abs(qty * rate - total) <= tol

Duplicate detection. Compare a content hash of the file, plus a fuzzy key such as (vendor, date, total, vehicle), against recent entries. Forwarded photos are extremely common.

Vehicle and trip matching. Match the extracted plate against your vehicle master. Then match the date against trips recorded for that vehicle. If two vehicles or two trips could match, do not pick one. Ambiguity is a review reason, not a coin toss.

Date sanity. Future dates or dates long before the trip get flagged.

4. The gate: auto-post or review

Only post automatically when every check passes. Anything else goes to a queue with explicit reasons:

def route(doc, fields, checks) -> tuple[str, list[str]]: reasons = [] if not checks["sender_authorised"]: reasons.append("unknown sender") if any(f.value is None for f in fields.values()): reasons.append("missing field") if min(f.confidence for f in fields.values()) < 0.90: # illustrative threshold  reasons.append("low OCR confidence") if not checks["arithmetic_ok"]: reasons.append("qty x rate != total") if checks["possible_duplicate"]: reasons.append("possible duplicate") if checks["match_count"] != 1: reasons.append("vehicle/trip match not unique") return ("auto_post", []) if not reasons else ("review", reasons)

A few practical lessons:

  • Show reasons to the reviewer. "Low confidence on vehicle number" is far faster to resolve than a generic "needs review".
  • Let supervisors pause automation. If a new pump's slip format starts confusing the extractor, someone should be able to switch off auto-posting rules without stopping intake. In JC Fleet, supervisors can pause and resume automatic posting rules at any time.
  • Allow retry or manual correction. Sometimes re-running extraction on a better crop fixes it. Sometimes a human just types the right number.
  • Corrections should be traceable. Store who changed what, and keep the original extraction.

5. Post with the original attached

When a record is created, whether automatically or after review, link it to the original document. That's what makes an expense auditable later. A trip settlement or vehicle P&L is only as trustworthy as the entries behind it.

Why WhatsApp-first (and no driver app)?

Drivers already use WhatsApp. Asking them to install and update a separate app adds friction, and adoption suffers. Meeting users in the channel they already use made a bigger difference than any OCR tuning. The same thinking applies beyond trucking: build intake around existing behaviour, then add structure behind it.

Takeaways

  1. Treat OCR as a draft. Gate auto-posting on explicit, testable checks.
  2. Cheap validations (arithmetic, duplicates, unique matches) catch a lot.
  3. Make review fast with clear reasons, the original image and retry or correct options.
  4. Keep the original document linked to every record.

If you're curious how this looks in a real workspace with Hindi and English bills, trips, tyres and ledgers, take a look at jcfleet.com. For custom AI, OCR or automation work in your own domain, the JC TechLabs team is at jctechlabs.com (contact@jctechlabs.com).