By ITRoad Group | Technical Case Study
The Problem Nobody Wanted to Touch
Every organization has that one process. The one that’s been running the same way for fifteen years. The one that works — technically — but only because a dedicated team of people shows up every morning and manually powers it through. Expensive, fragile, and quietly eating into operational budgets.
For one of our public sector clients, that process was document intake and data extraction.
Every day, thousands of incoming documents — forms, reports, submissions, scanned files — needed to be read, understood, classified, and have their key data extracted and routed into downstream systems. The team handling this was large. The work was repetitive. And while the people doing it were skilled, the process itself was fundamentally a bottleneck: slow to scale, prone to inconsistency, and increasingly difficult to justify at the cost it demanded.
When they came to us, the question wasn’t “can this be automated?” — it was “why hasn’t it been already?”
Why Legacy Automation Had Failed Before
This wasn’t the client’s first attempt at automation. They’d tried rules-based OCR systems. They’d experimented with template matching. None of it stuck.
The core challenge: document variability. Government intake processes deal with documents that don’t conform neatly to a fixed format. Submissions come from dozens of different sources, in different layouts, with different terminology, sometimes handwritten, sometimes scanned at poor quality. A system that works perfectly on one document type breaks on the next.
Traditional automation assumes structure. These documents had almost none that could be reliably predicted.
What was needed wasn’t a smarter rules engine — it was something that could read and understand documents the way a human does, at machine scale.
The Architecture: An Agentic Workflow
Our solution was built around a multi-step agentic pipeline — a system where a Large Language Model (LLM) doesn’t just respond to a single prompt, but acts as an orchestrator, breaking down a complex task into discrete steps and executing them in sequence with conditional logic at each stage.
Here’s how the workflow operates end-to-end:
- Ingestion & Pre-processing: Documents arrive through existing intake channels (email attachments, portal uploads, scanned batches). A pre-processing layer handles format normalization — converting PDFs, images, and scanned files into a clean text representation.
- Document Classification Agent: Before any data extraction happens, a classification agent reads the document and determines its type, origin, and processing pathway. Documents falling below a certain threshold are flagged for human review.
- Extraction Agent: Once classified, a dedicated extraction agent reads the document against a specific schema. It understands semantic intent — it knows that “date of submission” and “received” mean the same thing, regardless of layout.
- Validation & Confidence Scoring: Every extraction result carries field-level confidence scores. A validation layer cross-references extracted values against known reference data and flags anomalies.
- Human-in-the-Loop Review: We deliberately kept humans in the process as quality guardians. The review interface surfaces only the exceptions: low-confidence extractions or novel document types.
- Audit Trail & Compliance Logging: Every decision the system makes — classification, extraction, validation — is logged with the reasoning behind it for full transparency and regulatory requirements.
The Technology Stack
- LLM backbone: A frontier model with strong instruction-following and structured output capabilities.
- Orchestration: A custom agentic framework managing prompt chaining, state passing, and retry logic.
- OCR layer: A hybrid approach combining traditional OCR engines with vision-language models.
- Integration: RESTful APIs connecting to the client’s existing document management system — no legacy system replacement required.
Results
- ~80% reduction in manual headcount required for processing.
- Processing time dropped from 2–3 days to under 15 minutes.
- Extraction accuracy exceeded 95%.
- Exception rate stabilized at under 12%.
- Cost per document reduced by more than 60%.
Lessons Learned
- Start with the exception, not the rule: Invest effort in designing what happens when the system isn’t sure. That’s where trust is built.
- Confidence scoring is non-negotiable: An AI system without a certainty indicator is dangerous in an operational context.
- Don’t automate away domain expertise: We embedded staff knowledge into the classification and validation rules.
- Buy-in comes from transparency: Stakeholders drop resistance when they can see why the system made a decision.
Working on Something Similar?
At ITRoad Group, we specialize in designing and building agentic AI systems that connect to your existing infrastructure and deliver measurable outcomes.
Get in touch at itroadgroup.com