I've been conducting a series of systematic latency tests on Scholarcy's PDF-to-summary pipeline over the past month. My primary use case involves processing academic papers (typically 10-30 pages) via the browser extension and the API.
The inconsistency in processing times is significant. For a set of 15 similar-length PDFs from the same journal, I observed:
* **Minimum time:** 4.2 seconds
* **Maximum time:** 23.7 seconds
* **Average:** 9.8 seconds
* **Standard deviation:** 5.1 seconds
This degree of variance suggests non-deterministic backend behavior. I suspect it's related to either dynamic resource allocation or variable pipeline stages.
Has anyone else done controlled measurements? I'm particularly interested in isolating the variables. My preliminary hypothesis points to:
1. **PDF complexity:** Not just page count, but embedded figures, fonts, and mathematical notation.
2. **Network routing:** Which AWS/Azure region is handling the OCR or parsing workload?
3. **Queueing delays:** During peak hours (approx. 09:00-12:00 UTC), my 90th percentile latency spikes.
My test methodology for a single document was:
```javascript
// Simplified version of my custom logging wrapper
const start = performance.now();
const result = await scholarcy.processDocument(file);
const end = performance.now();
console.log(`Processing latency: ${(end - start).toFixed(0)}ms`);
```
I'd like to compare notes. Are the inconsistencies more pronounced on the API tier versus the browser extension? Has anyone traced the latency to a specific stage, like the initial PDF parse versus the summary generation?
sub-10ms or bust
Finally, someone with actual numbers instead of marketing fluff. That standard deviation alone is telling.
But your controlled methodology is the real issue. Are you using synthetic PDFs or live journal downloads? If it's the latter, you're introducing all the vendor's caching variables. Run the same PDF through five times in rapid succession and watch the variance collapse. It's their load balancers and warm containers, not your document complexity.
And queueing delays? Of course there are queueing delays. They're not giving you dedicated inference hardware at their listed pricing. The real question is whether that 90th percentile spike happens during your local business hours or theirs. If it's theirs, they're capacity planning for their own region, not yours.
Trust but verify.
Your point about vendor caching is well taken, but I think you're dismissing the document complexity variable too quickly. Even with warm containers, the pipeline stages for extraction and summarization can vary wildly based on PDF structure - a dense, multi-column layout with embedded vector graphics versus a simple text-based single column PDF will invoke different OCR and layout analysis pathways, even from the same warm instance.
The load balancer distribution is a factor, sure, but in my own migration playbooks for data processing systems, I've logged timestamps that show the initial PDF parsing step alone can have a 300% time delta across document types on identical hardware. Re-running the same PDF smooths the curve, but that only tests one document class.
The more useful benchmark would be to stratify by document complexity *and* run at consistent intervals against both cold and warm states. That would isolate infrastructure variance from processing variance. Without that, we're just guessing which component is the bottleneck.