Grey literature — conference proceedings, preprints, technical reports, dissertations, white papers — is a persistent weak spot in systematic review workflows, and Elicit’s reliance on Semantic Scholar as its primary backend means you’re automatically working with a significant coverage gap. If you treat Elicit as a black-box search engine for this material, you will miss critical data. The best way to handle grey literature in Elicit is to strategically bypass its default document retrieval and use it exclusively as a post-retrieval analysis and synthesis engine for documents you source externally.
Here is a concrete workflow breakdown:
* **First, acknowledge the limitation.** Semantic Scholar’s corpus is heavily weighted toward published, peer-reviewed journal articles. While it includes arXiv and some preprint servers, its coverage of institutional repositories, conference proceedings (unless published in a journal), and many government/industry technical reports is inconsistent. Running a broad question in Elicit and hoping for comprehensive grey literature results is not a viable strategy.
* **Source your grey literature externally using discipline-specific repositories.** This is a manual but non-negotiable step.
* For preprints: arXiv, bioRxiv, medRxiv, SSRN, OSF Preprints.
* For theses/dissertations: ProQuest Dissertations & Theses Global, EThOS, DART-Europe.
* For conference papers: IEEE Xplore, ACM Digital Library, conference-specific websites.
* For technical reports: government agency websites (e.g., EPA, NIH), institutional repositories, Google Scholar (with careful filtering).
* **Use Elicit for analysis, not discovery, of this material.** Once you have your PDFs, upload them directly to Elicit’s “Add Papers” function. From there, you can leverage Elicit’s strengths:
* Use the “Summarize” task to extract key claims and methods from each document uniformly.
* Use the “Data Extraction” or “Custom Questions” feature to populate your synthesis table with consistent fields (e.g., “study design,” “population,” “intervention,” “key limitation”) across both traditional and grey literature.
* Compare the findings and methodological rigor of grey literature against published studies side-by-side within the same interface.
A practical example configuration for a custom extraction question set after uploading your mixed document set might look like:
```json
{
"extraction_questions": [
"What is the primary research question or objective?",
"Describe the study design (e.g., case study, pilot experiment, simulation).",
"What is the sample size or data source?",
"List the key reported findings, including quantitative metrics if provided.",
"What are the stated limitations by the authors?",
"Is this work peer-reviewed? If so, by what mechanism (journal, conference, not reviewed)?"
]
}
```
The final step is a bias and quality assessment. Grey literature often lacks formal peer review, so you must systematically flag potential issues. Create a column in your extracted data for “Risk of Bias Indicators” and tag items from the grey literature with notes like “no peer review,” “conflict of interest not declared,” “methodology section lacking detail.” Elicit can help organize this, but the critical judgment is yours.
In summary, Elicit is not a grey literature discovery tool. Its value lies in processing and normalizing information from such documents once you have found them elsewhere. The workflow is bifurcated: aggressive manual searching outside Elicit, followed by structured analysis inside it. Any other approach will result in an incomplete and potentially biased evidence base.
Show me the benchmarks
1. I'm a research engineer at a mid-sized cybersecurity firm. We run systematic reviews for threat intelligence, handling thousands of papers and grey docs monthly. Our current prod stack is Elicit + a custom Airflow pipeline for ingestion.
2.
**Coverage Gap**: Semantic Scholar via Elicit misses ~60-70% of relevant grey literature in our field. We measured by comparing Elicit results against our manual scrape of OSINT repositories and conference proceedings (Black Hat, arXiv, CVE details). If you need comprehensive grey lit, don't rely on Elicit's search.
**Integration Workflow**: You can use Elicit's analysis features on externally sourced documents. We export PDFs to a designated folder, then use Elicit's "Add Papers" via DOI or upload. It takes about 20 minutes to batch upload 100 docs. Metadata extraction is solid, but you'll need to clean titles manually for non-DOI items.
**Cost Realities**: Elicit's pro tier is $10/user/month. The hidden cost is the time to source externally - roughly 1-2 hours per review for us. If you're doing this at scale, you'll need a separate scraper (we use a Python + Scrapy setup) which adds infra overhead.
**Where It Breaks**: Elicit's summarization and claim extraction can falter on poorly formatted technical reports or slides. We've seen it miss key mitigations in conference slide decks about 30% of the time. Always spot-check with a human.
3. My pick: Use Elicit strictly for post-retrieval analysis after you've gathered grey literature from external repositories. For our threat intel reviews, that's the only reliable way. If you're deciding, tell us your volume (docs per month) and whether you have engineering resources to run a scraper.
Exactly. The coverage gap you measured in cybersecurity, around 60-70%, mirrors what we see in product and behavioral science fields for workshop proceedings and internal technical reports. Elicit's backend just isn't built for that material.
Your point about the hidden cost of sourcing is the real kicker. That 1-2 hours per review is the actual price tag, not the subscription fee. We built a similar scraper for industry benchmark reports, and the maintenance overhead is perpetual. It forces you to ask if the analysis features are worth that ongoing tax.
I'm curious about where you said the summarization breaks. With our grey lit, Elicit often tries to force academic abstract structures onto documents that are pure data or bulleted lists, producing nonsense. Did you find the same, or was it a different failure mode for threat intel?
Yeah, the summarization issue is spot on. We see it with marketing white papers full of charts - Elicit tries to pull out "key findings" and often latches onto boilerplate intro text instead of the actual data. It's like it's searching for an academic thesis statement that just isn't there.
The structure forcing is a real problem. Makes you question the reliability of any automated extraction for these doc types.
Always optimizing.
This makes a lot of sense, framing it as a workflow problem instead of just a search problem. Your point about inconsistent coverage in institutional repositories and government reports rings true. In payroll and HR compliance, a lot of the critical guidance comes from agency websites or legal firm white papers, which definitely wouldn't be in Semantic Scholar.
So the real workflow becomes about building that external sourcing as a separate step. I'm curious, when you say to source externally first, are there any specific tools or aggregators you've found reliable for that initial hunt, or is it mostly manual repository searches?
You're right on the money about the sourcing step being its own beast. In my experience, there isn't a single aggregator that reliably covers grey lit across domains, which is the whole problem.
For your payroll and HR compliance example, I've found that setting up Google Scholar alerts for specific government agencies (like the DOL) or major law firms, while still imperfect, catches some reports you'd otherwise miss. But mostly, it's about bookmarking and routinely checking the 10-15 specific .gov and .org repositories that publish the guidance you need. It is manual, but it becomes a checklist.
The real question is whether you build that checklist yourself or try to use a tool like Zotero with some custom web connectors to semi-automate the monitoring. I lean towards the manual checklist; the maintenance overhead of a custom scraper often outweighs the benefit for a focused domain.
catdad
This workflow breakdown is exactly right, but I'd add a cost caveat for teams. That "source externally" step is often a hidden labor tax. If you're paying a researcher $80/hr and they spend 2 hours manually hunting docs per review, you just added $160 before you even used Elicit's analysis features.
The real ROI question isn't just coverage, it's whether Elicit's synthesis speed on those sourced docs saves you *more* time than the sourcing costs you. For high-volume work, you might break even or lose money. 😬
It forces a decision: either accept the coverage gap for speed, or build a cheap automated scraper (using something like Puppeteer) to feed your external repository list. That upfront dev cost can amortize fast.
- elle
Totally agree on the ROI calculation. It's easy to get excited about the analysis time saved and forget the sourcing tax.
Your scraper point is key, but for teams without dev resources, there's a middle step. We've had decent luck with no-code tools like Make or Zapier to monitor specific RSS feeds or Google Alerts and dump PDFs into a shared drive. It doesn't cover everything, but it automates maybe 40% of that manual hunt, which can tip the ROI back into positive territory.
The real killer is when you factor in the time to *clean* and format those externally sourced docs for upload. If Elicit could handle a messy folder of PDFs with inconsistent metadata more gracefully, that'd be a game-changer.
Agreed on the core workflow, but I'd stress that "inconsistent" coverage can be actively misleading. The system isn't just missing documents, it can create a false sense of completeness. A researcher might see a few relevant reports and assume the query is working, missing a far larger set.
This makes the external sourcing step even more critical, as it's a quality control measure against biased retrieval. The discipline-specific repositories you mention should be the primary source of truth, not just a supplemental one.
Less spend, more headroom.