We’ve been working with a large collection of old newspapers and scanned editions from the early 80s through the 2000s. While everything has technically been “digitized,” most of it is locked up in static PDFs or low-quality image files. The real issue wasn’t OCR, it was everything after that.
For example, we struggled with articles broken across pages, headlines buried in noisy layouts, and visual clutter from old ads and classifieds. Even if the text is extracted, it’s tough to figure out what’s a coherent story and what’s just noise. On top of that, we’ve got non-English content and irregular formatting depending on the year and the publication.
Recently, we’ve been trying out an AI-based Digital Archive Software. It uses a mix of NLP and layout detection to parse out actual articles, extract metadata like date and section, identify named entities, and make the content searchable by meaning, not just keywords. It's definitely not a hands-off process, some manual validation is still part of it, but the software handles a lot of the grunt work that would be nearly impossible to scale otherwise.
Would be curious to know how others are solving this. Are you building custom pipelines? Using layout-aware models? How do you handle multi-language archives or handwritten scans?
For example, we struggled with articles broken across pages, headlines buried in noisy layouts, and visual clutter from old ads and classifieds. Even if the text is extracted, it’s tough to figure out what’s a coherent story and what’s just noise. On top of that, we’ve got non-English content and irregular formatting depending on the year and the publication.
Recently, we’ve been trying out an AI-based Digital Archive Software. It uses a mix of NLP and layout detection to parse out actual articles, extract metadata like date and section, identify named entities, and make the content searchable by meaning, not just keywords. It's definitely not a hands-off process, some manual validation is still part of it, but the software handles a lot of the grunt work that would be nearly impossible to scale otherwise.
Would be curious to know how others are solving this. Are you building custom pipelines? Using layout-aware models? How do you handle multi-language archives or handwritten scans?