Just migrated from RefWorks to SciSpace. Export was a .bib file, import seemed fine. Now I have 378 "duplicate" entries of my 124 actual references.
SciSpace's duplicate detection appears to be checking the *filename* of the attached PDF, not the DOI or title. Since my local files are named like `author_year.pdf`, every single one is a duplicate.
* Their algorithm is clearly broken for bulk imports.
* No option to force merge or skip duplicate check on import.
* "Smart" feature? More like lazy string compare.
Anyone else hit this? Is there a hidden CLI import tool or some JSON voodoo to bypass this, or am I stuck manually cleaning up their mess?
Tried re-exporting from RefWorks in RIS format. Same result.
-- old school
-- old school
Ah, the classic case of a "smart" feature built on a single, brittle heuristic. PDF filename matching for duplicate detection in a reference manager is a new level of misguided optimization.
You're probably stuck with manual cleanup, but before you resign yourself, try this nuclear option: strip the PDF attachments from the .bib or RIS file before import. A quick sed or Python script can remove the `file = {...}` lines. The import will be "broken" in their eyes, but at least you'll get your 124 unique entries. You can reattach the PDFs later via their UI, painfully, one by one, as a form of penance for using a system that thinks a filename is a canonical identifier.
It's the kind of architectural decision that screams "we needed a Jira ticket closed by Friday."
keep it simple
Worse, that brittle heuristic becomes a locked-in "feature". Now any batch operation you try to script will have to work around it, creating a permanent tax for using their platform. The nuclear option is clever, but you're just accepting that the system's core logic is something you must avoid. That's not a tool, it's an obstacle with a UI.
Your vendor is not your friend.