I've been using Elicit to scope literature for infrastructure-as-code research, which is heavy on technical papers and benchmarks. It's decent for that. Recently, a colleague in a social science department asked me to help evaluate it for their qualitative research workflow. The results were mixed, exposing some clear gaps.
The core issue is that Elicit's strengths are pattern-matching and extraction from structured, quantitative findings. Qualitative studies don't play to those strengths. Here's what we observed:
* **Search relevance is lower.** Searching for themes like "community adoption of technology" returns many quantitative surveys on adoption rates, but misses nuanced ethnographic work unless the title/abstract explicitly states the methodology.
* **Column extraction is often useless.** The automatic columns for "population," "intervention," "outcome" are frequently empty or misapplied. You can't teach it to extract "research method," "theoretical framework," or "key themes" from the PDF text.
* **It misses the "why."** The one-sentence summaries sometimes capture the study's conclusion but strip out the methodological approach and depth of analysis, which is the critical factor for qualitative researchers.
We tried a workaround: using very specific search strings with methodology terms ("ethnography", "case study", "grounded theory", "phenomenology"). This improved the initial paper retrieval, but Elicit's analysis tools still couldn't engage with the content meaningfully. It's like having a powerful parser that's configured for the wrong schema.
Has anyone else pushed Elicit into this type of literature review? Specifically:
* Have you found effective prompts or workflows to surface qualitative work better?
* Did you combine it with other tools (like systematic review software) to compensate?
* Is the upcoming "upload your own papers" feature capable of better extraction if you feed it a curated set?
plan before apply
Good observations on the column extraction. I've seen similar mismatches in my field, where it forces studies into a rigid "intervention/outcome" mold even when that doesn't fit.
For the "missing the why" point, that's critical. A tool optimized for extracting a measurable result will inherently flatten qualitative analysis. It's not just a gap in the algorithm, it's a mismatch in design goals. I wonder if the underlying model was primarily trained on biomedical or STEM abstracts where the structured abstract format reigns.
Have you found any workarounds for the search relevance issue, like using very specific methodology terms in the query?
Mike D.
Your colleague's experience is the whole problem in a nutshell. The tool wasn't built for the task, and no amount of prompting will fix a fundamental design mismatch.
The real cost isn't just the missed papers. It's the false confidence that your search is comprehensive. You'll waste more time verifying gaps than you saved.
Look for tools built by qualitative researchers, even if they're less polished. Or just accept that for qualitative work, there's no substitute for actually reading the abstracts.
Read the contract