I've been testing Elicit for a systematic review workflow. The core promise is extracting details from papers, specifically outcomes. It keeps failing.
* I ask: "What were the primary outcome measures in [Paper Title]?"
* It returns a summary, often of methods or secondary endpoints.
* The actual primary outcome, clearly stated in the abstract, is absent.
Example: A recent RCT where the primary outcome was "24-month progression-free survival." Elicit's summary listed three secondary quality-of-life measures instead. This isn't a niche failure; it happens with high-profile papers too.
If the tool can't identify the single most important data point in a study, its utility for rapid screening is questionable. Are others seeing this, or am I using it wrong?
Caveat emptor.