CASE STUDY · EXTRACTION
From scanned page to structured record
A hierarchical semantic-segmentation pipeline with rule-based post-processing that extracts key-value pairs and tables at 90% accuracy — and cut manual intervention in extraction workflows by 30%.
At a glance
- role
- R&D ML Engineer, Ninestars (led)
- result
- 90% extraction accuracy
- ops impact
- −30% manual intervention
- approach
- hierarchical segmentation + rules
- stage
- full-scale production
Problem
Classification tells you what a document is; the business value is in what it contains. Key-value fields and tables live at unpredictable positions across layouts and scan qualities, and manual extraction doesn’t scale — humans were the bottleneck in the document workflows.
Approach & architecture
A two-stage design:
- Hierarchical semantic segmentation — models segment the page top-down into progressively finer regions, localizing where keys, values, and table structures live.
- Rule-based post-processing — deterministic rules turn segmented regions into clean key-value pairs and tabular records, catching what pure learning gets wrong and keeping outputs auditable.
Around the models, I automated the surrounding document data-extraction workflows end to end, which is where the 30% reduction in manual intervention came from.
The hard part
Hybrid systems earn their keep at the boundary: deciding what the segmentation model should own and what the rules should own. Learned segmentation handles layout variety; rules keep precision and auditability on the structured output. Tuning that boundary — per field type, against real production documents — was the core of the work.
Result
90% accuracy on key-value and tabular extraction in full-scale production, and 30% less manual intervention in the extraction workflows. Team-wise, I led the data annotators producing training data and coordinated front-end, UI/UX, and DB colleagues to ship it.
Stack
- Python
- PyTorch
- U-Net (segmentation)
- OpenCV
- SQL
- MongoDB
stack mapping from skills inventory