Okay, let's get spicy. I've been living in Elicit for the past three months, using it for a deep dive into PLG adoption metrics for a side project. And I've reached a conclusion that's been bugging me: **Elicit is dangerously good at creating the *illusion* of rigorous research, which might be worse than no research at all.**
Here's my experience. When you first fire it up, ask a question, and get that neat table of papers with summaries and interventions, it feels like magic. You feel productive! You've "done your literature review" in an hour! But that's the trap. If you don't already have a solid grounding in your field and a sharp, critical question, Elicit will happily feed you a pile of plausible, decontextualized snippets. You can string these together into something that *looks* like a paper, with citations and everything, but it's fundamentally surface-level. It's a paper mill because it efficiently assembles parts, not because it builds understanding.
I think the problem lies in the workflow it *unintentionally* encourages for beginners:
* **The "Question-as-Google-Search" Pitfall:** You ask something broad like "What causes churn in SaaS?" Elicit gives you 50 papers. You skim the summaries, pick a few that sound relevant, and cite them. But you've missed the key debates, the methodological flaws in the 2012 study it surfaced, or the seminal 2015 paper it *didn't* surface because of its keyword matching.
* **Summary Over-Reliance:** The abstract summaries are a fantastic time-saver *if* you're using them to triage which papers to read deeply. If they're your *primary source*, you're building on a second-hand interpretation. I've seen summaries subtly miss a study's limiting condition or overstate a finding.
* **The Illusion of Comprehensive Coverage:** It's easy to think "Elicit found everything." But it's working with what's in Semantic Scholar. Gaps exist. If you don't know the key authors or landmark studies in your field already, you won't know what's missing from Elicit's results.
This isn't Elicit's fault—it's a tool, and a powerful one. My hot take is that **Elicit amplifies existing research skill.** If you're a novice, it amplifies your naivete, letting you generate shallow work faster. If you're an expert, it amplifies your efficiency, letting you test hypotheses and find connections at speed.
The real value for me kicked in when I started using it *after* I had done some old-school reading and had specific, narrow queries:
* "List studies published after 2020 that measure the impact of [Feature X] on [Metric Y], excluding studies focused on enterprise contexts."
* "Find papers that cite both [Author A] and [Author B]."
* "What are the main criticisms of [Specific Theory]?"
Used this way, it's incredible. But I worry about students or eager PMs jumping in, thinking the tool does the thinking for them. It's the ultimate "garbage in, garbage out" engine, and it's so polished that the garbage can look… citation-worthy.
Anyone else feel this tension? How do you structure your Elicit workflow to force depth and avoid the paper-mill vibe?
🔥
Try everything, keep what works.