That's really concerning. Fabricated papers are the worst kind of error because they're so hard to catch if you're not already an expert in that specific academic niche.
It makes me wonder about the training data for these tools. If it's just pattern-matching language from the internet, of course it will invent plausible-sounding citations. There's no actual database of papers to pull from.
I've had similar issues with AI suggesting email marketing strategies that reference non-existent features in platforms I use. It sounds right until you try to execute it. Does SciSpace at least warn users that citations might be made up?
> "There's no actual database of papers to pull from."
Exactly. It's pattern completion, not retrieval. The citations are just the most statistically likely word salad that follows "et al. (2023)" in its training data.
Your email marketing example is the same failure mode, just in a different domain. These tools have no connection to a live API or a current feature set. They're synthesizing language from past documentation and forum posts.
As for warnings, most bury a tiny disclaimer somewhere in their terms. But the demo is always the confident, flawless output. The onus of verification, like checking an AWS bill for missing credits, is silently passed to the user. You're not just using a tool, you're accepting a liability transfer. Fun, right?
- elle
That liability transfer is exactly what happens with infrastructure-as-code templates. An AI can generate a perfect-looking CloudFormation stack that uses `DependsOn` attributes in a logically sound way, but references resources from a service that was sunset last year.
The structure passes a linter, the syntax is valid, but it deploys nothing. You're left holding the broken deployment while the tool disclaims all responsibility in its tiny footer link.
It's pattern completion without a connection to the actual, current state of the API world, just like the citation database.
Commit early, deploy often, but always rollback-ready.