Hey folks, been putting Scholarcy through its paces on some longer academic papers and technical manuals for a project. It's fantastic for quick summaries on shorter docs, but I've hit a consistent performance wall with PDFs over roughly 50 pages. The processing just crawls, sometimes timing out entirely.
I'm trying to reason about the bottleneck from an architectural perspective. A few guesses:
* **Chunking strategy:** Parsing and sending the entire PDF to their API in one shot would be a killer. Are they chunking it client-side or server-side? If it's server-side, the upload of a large file alone is a bottleneck.
* **Text extraction overhead:** The PDF-to-text conversion for large, complex documents (especially with multi-column layouts, figures, tables) is computationally heavy. This might be happening before any AI summarization even begins.
* **Sequential processing:** If they're processing pages or sections sequentially in a single thread, rather than with some parallelization, it would explain the linear slowdown with page count.
From a backend dev viewpoint, handling large PDFs efficiently is tough. You'd ideally want:
* A streaming upload to avoid memory issues.
* Parallel text extraction and analysis stages in a pipeline.
* Maybe even a different, lighter model for initial chunking vs. final summarization.
Has anyone else experienced this or found a workaround? I'm considering a pre-processing step locally to split large PDFs into sub-50-page chunks before sending them through, but that defeats the convenience a bit. Would love to hear if the Scholarcy team has commented on their scaling approach for larger documents.
--builder
Latency is the enemy, but consistency is the goal.
Your guesses are solid. The text extraction phase is often the hidden bottleneck people overlook, especially with complex layouts. Even a high-quality parser like PDFPlumber or a commercial service needs to reconstruct reading order, which is computationally expensive and scales with page complexity, not just page count.
I'd add that their pricing model might be a factor. If they're charging per "summary" and processing the whole document server-side, they have an incentive to minimize their compute time. Efficient parallel processing costs more to build and run. You might be seeing a deliberate trade-off to keep their base subscription price low.
The timeout suggests they might have a hard cap on processing time per request. For anything over that threshold, you'd need a different approach, like pre-chunking the PDF yourself before uploading sections.
You're right about the layout reconstruction being a killer. We had to build a custom middleware pipeline for a client pulling text from thousands of engineering spec PDFs, and the reading order logic from even the best libraries would sometimes just give up on dense pages, consuming CPU for minutes before failing. It's not linear at all.
The pricing model angle is interesting, but I'd flip it. I suspect the timeouts aren't a deliberate cost-saving throttle, but a crude failure mode of their architecture. A well-designed async queue for long-running parse jobs wouldn't just time out; it'd give you a webhook or a job ID to poll. That they don't suggests they're hitting a hard limit on a synchronous request handler, probably behind something like API Gateway with a 30-second limit. That's an infrastructure choice, not a pricing one.
Pre-chunking is the only reliable workaround I've found for services like this. You end up writing a pre-processor that splits the PDF by logical sections (like chapters) before sending, which of course adds its own layer of complexity and edge cases.
APIs are not magic.