Hi everyone,
My team (we're a group of 10 researchers and analysts in public health) recently made the switch from relying primarily on Semantic Scholar to using Elicit for our literature reviews. We'd been using Semantic Scholar for quick checks, but needed something that could handle more systematic screening and data extraction. After a 3-month pilot, I wanted to share our experience, especially from a data workflow perspective.
**The main reason for the switch was the bulk processing.** Uploading a CSV with 500+ paper titles/abstracts from our initial searches into Elicit and having it summarize each one against our specific research questions was a game-changer. It cut our initial screening time down by about 60%. Semantic Scholar is great for discovery, but we found Elicit's interface for managing and categorizing large sets of papers to be much more aligned with a team review process.
Here’s a simplified version of the workflow we built:
1. Export initial search results from PubMed/Scopus to CSV.
2. Upload CSV to Elicit using the "List of papers" task.
3. Use a custom prompt to extract key details. We standardized on asking for: Population, Intervention, Outcome, and Study Design (PICO-S). This gave us a structured table to export.
The export to CSV is clean, which made it easy to then load into our shared SQL database for the final team decision tracking. I set up a simple table for the results:
```sql
CREATE TABLE elicit_screening_results (
paper_id INT PRIMARY KEY,
original_title TEXT,
elicit_summary TEXT,
population TEXT,
intervention TEXT,
outcome TEXT,
study_design TEXT,
team_decision VARCHAR(50) -- 'include', 'exclude', 'maybe'
);
```
**Some hiccups we encountered:**
* The quality of the extraction is *highly* dependent on how you phrase the prompt. We had to iterate a few times to get consistent PICO-S extractions.
* It sometimes "hallucinates" details that aren't in the abstract, so we treat its output as a high-quality screening aid, not a final answer. A second human check is still mandatory for us.
* The cost can add up for very large batches, so we now do a quick deduplication in SQL before uploading.
Overall, for a team our size, Elicit has become a crucial tool for the initial heavy lifting. It hasn't replaced our critical thinking, but it has made the first, most tedious part of the systematic review much more manageable. Curious if other small teams have built similar pipelines or have tips for prompt engineering in academic contexts!