Let's get this out of the way: Elicit is excellent for what it is, which is a structured ideation engine. Where people fail is expecting it to be an autonomous research librarian that spits out digested truth. It's not. It's a very smart, very fast way to map a conceptual space *before* you do the actual work.
I've used it to kickstart threat model brainstorming for new cloud deployments. You give it a prompt like "academic papers on zero trust for microservices in multi-cloud," and the resulting matrix of papers gives you a scatter plot of the landscape. This is invaluable.
The critical pitfalls, however, are severe if misunderstood:
* **The "Summary" is a starting point, not a conclusion.** It's an aggregation of abstracts, often missing crucial nuance. I've seen it blissfully conflate conflicting study results. You *must* trace back to the source.
* **It can create a false sense of completeness.** The list looks exhaustive, but it's only as good as its underlying corpus. Niche or very recent incident response postmortems? Often missed.
* **It encourages lazy querying.** Vague questions get vague, shallow results. The skill is in iterative, specific prompting. For example:
```
// Weak
"Papers about cloud cost optimization"
// Better
"Empirical studies on unexpected cost drivers in serverless architectures (AWS Lambda, Azure Functions) published after 2020"
```
Treat it like a turbo-charged, semi-random literature search. It gives you a list of leads to then go and **read properly**. Using it as a final source is like trusting a security scan without reviewing the CVEs yourself. A fast path to a compliance finding.
- Nina
- Nina
Agree on the core function, but you're downplaying the vendor risk. It's a "structured ideation engine" until you've ingested a few hundred papers through it and realize your team's entire pre-research workflow is now dependent on their API pricing and uptime.
The real pitfall is when these tools become gatekeepers to your own institutional knowledge. What's their data retention policy? If you stop paying, do you lose your query history and those curated "landscape" matrices? That's the lock-in they don't advertise.
Trust but verify.
Your threat modeling example is spot on, but it reveals another hidden cost. That initial scatter plot can lead you down a path of architecting for threats that are statistically irrelevant or have a mitigation cost far exceeding the risk.
You used it for multi-cloud zero trust. How many of those papers were conceptual vs. case studies with actual cost breakdowns? The tool might lead you to implement an overly complex policy framework when a simple, well-configured VPC endpoint would address 80% of the risk at 10% of the operational overhead. The mapping phase needs a cost filter before any architecture decisions get made.
Less spend, more headroom.