Hi everyone,

I’m looking for advice from people who have experience building AI-powered document understanding systems, particularly around PDF semantic tagging and document structure extraction.

Over the past four months, I’ve spent well over 100 API credits experimenting with different LLMs and prompting strategies to build a platform that can generate high-quality semantic tags for PDFs.

The goal is not simple OCR or text extraction. I want the AI to accurately identify and tag document elements such as:

The benchmark I’m trying to reach is Adobe’s PDF Accessibility/Tagging quality, but despite trying multiple approaches, none of the current models I’ve tested consistently achieve that level of accuracy.

Some of the things I’ve already experimented with include:

Unfortunately, the results are still far from production quality, especially for complex PDFs containing multi-column layouts, nested tables, figures, and mixed formatting.

At this point, I feel I’m missing an architectural approach rather than simply choosing a better model.

I’m looking for people who have experience with:

I’d really appreciate insights on questions like:

• Is an LLM alone the wrong approach?

• Should this be built as a multi-model pipeline?

• Are there open-source models that perform better for document layout understanding?

• How would you architect a production-quality PDF tagging system?

• Has anyone successfully achieved results close to Adobe’s implementation?

I’m not looking for someone to solve the problem for me—I genuinely want to learn from people who have tackled similar challenges.

If you’ve worked on document AI, I’d love to hear about your approach, lessons learned, or even papers, repositories, or architectures worth exploring.

Thanks in advance!