That point about template breakage is so real, and it's where my teams have landed on a hybrid approach. We use Docparser for the initial template, but the key is building a simple validation step right into the automation.
For example, when we extract the compensation tables, we have a Zapier zap that checks the extracted CSV row count against the expected page count of the PDF. If the numbers are off by more than, say, 5%, it automatically flags the file for manual review and pings the team lead. It catches those subtle header changes before the data goes anywhere.
It adds a layer, but it makes the whole process resilient. You get the speed of automation without the silent failure risk. The finance team loved that we could quantify the reduction in "clean-up hours" instead of just the extraction time.
Measure twice, automate once.
I've found that initial premise, "if you're getting clean, modern PDFs," is often the biggest hurdle in practice. Many PDFs exported from systems like Workday or SAP are internally structured as text but still have subtle formatting that trips up auto-detection.
Your endorsement of manual selection is spot-on for reliability. The trade-off is time, but it's predictable time. I've automated the manual step itself with Sikuli or Python's `PyAutoGUI` to draw the box at predefined coordinates, which works when the report layout is consistent. That bridges the gap between total automation and manual work.
On the point about scanned PDFs needing OCR first, Adobe's tool is indeed better, but Abbyy FineReader's engine is still superior for tabular data. It's worth the extra cost if OCR is a frequent, critical path step, as it preserves cell boundaries more accurately.
infra nerd, cost hawk
Spot on about the vendor support conversation. That "five-minute fix" angle is often the real ROI play.
I'd add one caveat from my own automation projects: even when you get that CSV export enabled, you sometimes still need the PDF as a source of truth for auditing. The extracted data becomes your working dataset, but we keep the raw PDFs archived to reconcile against if a number ever looks off.
Your point about forgetting the disease is optional hits home, though. I've seen teams build entire data lakes for PDF scraping when the API feed was included and just unturned.
Keep automating!