I've been conducting a thorough evaluation of ResearchRabbit for potential integration into a reproducible research pipeline we're architecting, and I've hit a consistent, critical snag: the DOI metadata ingestion appears to be fundamentally unreliable for a non-trivial subset of academic papers. This isn't a matter of occasional lag; it's a systematic data integrity issue that, if uncorrected, calls into question the platform's utility as a source of truth for literature management.
The problem manifests in several distinct patterns I've documented over the past month:
* **Author list truncation or substitution:** A paper with six authors will be listed with only the first three, or occasionally, the entire list is replaced with a seemingly random author from a different paper. This breaks any workflow relying on accurate attribution or author-based filtering.
* **Publication date mismatches:** The date ingested via DOI often defaults to the online publication date from the publisher's page, while the PDF (or the platform's own card) shows the print/journal issue date. This creates chronological dissonance in sorted collections.
* **Journal title inconsistencies:** The most severe case I encountered was a DOI for a *Nature Communications* article resolving within ResearchRabbit to a completely different journal title, an obscure conference proceeding from over a decade ago. This appears to be a cache poisoning or upstream data retrieval error.
From an infrastructure-as-code perspective, this is akin to a Terraform module pulling in a wildly incorrect AMI ID from a supposedly stable registry. The input (DOI) is a standardized, unique identifier, but the output (metadata) is non-deterministic. For example, a simple metadata check would reveal:
```json
// Expected metadata (from CrossRef API)
{
"DOI": "10.1145/123456.789012",
"title": "Correct Paper Title",
"author": [{"family": "Actual", "given": "Author"}],
"published": {"date-parts": [[2023, 7, 15]]}
}
// Observed metadata in ResearchRabbit (hypothetical)
{
"title": "An Unrelated Paper",
"author": [{"family": "Wrong", "given": "Scholar"}],
"published": "2015"
}
```
My current hypothesis points to either an overloaded or misconfigured metadata caching layer, or an upstream data aggregation service they rely on that is introducing corruption. The issue is not universal, which makes it more insidious—it degrades trust gradually.
Has anyone else performed a systematic audit and encountered similar findings? More importantly, have you discovered any reliable workarounds beyond manual correction of every entry? I'm evaluating whether to build a pre-processing Lambda that validates all DOIs against a direct CrossRef/DataCite API call before import, but that negates much of the platform's promised efficiency.
--from the trenches
infrastructure is code