I've been evaluating SciSpace (formerly Typeset) as a potential reference management core for our research team's workflow, and I've hit a significant, reproducible data corruption issue during migration. The scenario: attempting to import a moderately sized EndNote library (approx. 2,500 references) using the `.enlx` export package. The import process completes, and the metadata (authors, titles, journals) transfers correctly. However, roughly 50% of the attached PDFs are either missing from the SciSpace library entirely or are present but corrupted—attempting to open them results in a "file damaged" error.
This is a critical failure for a tool claiming to be a research repository. A manual file-by-file audit confirms the source PDFs in the `.enlx` archive are intact. The corruption appears systematic, not random. I've replicated it three times with the same library.
My current diagnostic points to one of two potential failure modes in SciSpace's import pipeline:
1. **Path Length or Character Encoding Issue:** The attachments that fail often have longer file paths or special characters (parentheses, accented characters) in the original EndNote attachment link. SciSpace's import routine might not be handling these gracefully, truncating or corrupting the file during the copy/upload process.
2. **Batch Processing Timeout:** The import is a bulk operation. It's possible SciSpace's backend is timing out on individual file processing for larger attachments, leaving partial files in the database, but still marking the item as "imported."
**Has anyone else encountered and, more importantly, *solved*, this?** I'm looking for a systematic fix, not manual re-upload of hundreds of files. My immediate questions for the community or SciSpace support are:
* Is there a validated, step-by-step pre-processing ritual for EndNote libraries before import (e.g., must you use the "Copy Attachments" function in EndNote to a flattened directory first)?
* Does SciSpace provide import logs that detail *which* files failed and *why*? I've found none in the web UI.
* Is the official guidance to simply avoid the bulk import and add attachments manually? If so, that's a fundamental flaw in the migration value proposition.
From an infrastructure perspective, this smells like a classic case of insufficient error handling and idempotency in a data migration service. The lack of granular logging is particularly concerning for a tool that needs to handle academic data integrity.
Any detailed workflow reports or config tweaks that yielded a 100% successful transfer would be invaluable.
Path length and encoding issues are classic data migration gremlins. I've seen cloud billing manifests choke on parentheses in filenames and corrupt entire cost reports.
Your diagnosis is solid. Try this: create a small test library in EndNote with one "good" attachment (short ASCII name) and one "bad" (long path with accents or parentheses). Export to .enlx and run it through SciSpace's importer. If only the "bad" one fails, you've isolated the bug.
Might be easier to script a pre-migration cleanup of your EndNote attachments before the import. Rename files to strip special chars and flatten directory structures. It's annoying work, but less annoying than rebuilding half a library.
- elle
Good point about path length. I've seen similar issues where the import process doesn't handle the XML-based .enlx structure correctly - it might be reading the file path from the XML but then failing to decode URL-encoded characters or truncating the string when copying the binary payload.
The pre-migration cleanup script is the right call. You can automate it with something like a Python script that iterates through the EndNote library's `Data` folder, renames files to a safe hash-based name (e.g., `SHA1.pdf`), and updates the corresponding `.enl` library file references. It's a bit of a detour, but it isolates the problem to their importer's handling of metadata.
sub-100ms or bust