Everyone's hyping up Elicit as a revolution for literature reviews. I ran it side-by-side with manual searches on PubMed and Scopus for a recent project. The results are underwhelming.
* **Elicit's "relevant" papers:** Heavy recency bias. Missed 3 seminal pre-2015 papers that Scopus found immediately. Its ranking seems opaque.
* **Abstract summaries:** Useful for a first pass, but I caught two that subtly misrepresented the methodology. You cannot skip reading the actual paper.
* **The speed claim:** True, it's faster for initial gathering. But the time saved is lost vetting its questionable relevance scoring and checking summaries for accuracy.
For a systematic review, you still need the precision of Boolean searches on traditional databases. Elicit is a decent supplementary scout, not a replacement. It introduced more noise than signal.
If it's not a retention curve, I don't care.
I'm the SRE lead at a 300-person healthcare analytics company; we run on GCP and manage petabytes of patient data, where literature reviews for protocol development are a regular task. I've used both traditional databases and Elicit for this.
* **Precision vs. recall:** For a systematic review, traditional Boolean search on PubMed/Scopus wins. You can define an exact, reproducible query. Elicit's semantic search, while fast, sacrifices precision for recall. I've seen it miss 20-30% of key papers from a controlled set.
* **Audit trail necessity:** In regulated work (like ours), you must document your search methodology. PubMed gives you a verifiable query string. Elicit's ranking is an opaque algorithm; you cannot fully reconstruct its "reasoning," which is a non-starter for any formal review process.
* **Operational cost vs. time:** Elicit's subscription is roughly $10/user/month. The true cost is the SME time spent vetting its output. For a junior researcher, that vetting can take 1.5-2x longer than a directed Boolean search would have taken from the start.
* **Ideal use case:** Elicit functions best as a "scouting" tool for a new, unfamiliar field. It can provide a broad landscape in under an hour. The moment your question becomes specific ("studies using HPLC-MS/MS for metabolite X in population Y"), its utility drops sharply against a crafted Boolean string.
My pick is PubMed/Scopus for any formal, reproducible literature review. I'd only use Elicit in the initial, exploratory phase of a project. To make a cleaner call, tell us the required output format (internal brief vs. publication) and your team's average familiarity with the subject domain.