Elicit or Scite for meta-analysis in social sciences?

You've identified the core problem with both: they're built to sell subscriptions, not to efficiently complete a meta-analysis. Elicit is particularly egregious for burning credits on basic query refinement.

The real answer, unfortunately, is closer to your last point. For extraction, I've found a combination of Zotero with the Better BibTeX plugin and a well-structured spreadsheet is still the most reliable and cost-effective method. The "AI" tools often hallucinate numbers or miss key data points in PDFs, requiring you to verify everything anyway, which negates the time savings.

If you must choose, Scite is marginally more useful for checking citation contexts, but it's a luxury, not a workhorse. The grunt work is still manual.

You're right about the verification step eating up any time savings. I ran into that exact issue last month - used a tool to extract effect sizes from a batch of PDFs, and the "confidence scores" were high, but a spot-check found swapped control/treatment group numbers in three papers. Had to redo everything manually anyway.

That said, calling the manual Zotero/spreadsheet method "cost-effective" only holds if your time has zero cost. For a full-time researcher, maybe. But for anyone juggling this with other work, the sheer tedium has a real cost in morale and opportunity. I'd pay a reasonable fee for something that could reliably automate even 60% of the extraction, but you're right, neither Elicit nor Scite seem to hit that reliability bar yet without a crazy price tag.

The dream is a tool that works like a spell-check for data extraction: flagging potential numbers and asking "is this the n for group A?" instead of trying to do it all silently.

You've perfectly diagnosed the pricing pain points. The expired credits on Elicit are a critical flaw for any iterative research process. You don't just run one perfect query. You refine, test synonyms, adjust filters. That's dozens of credit-burning operations before you even get usable results.

On the extraction question, neither tool's core model is engineered for the structured data pull you need. They're built for semantic search and citation context. Trying to force them to consistently identify "sample size" or "Cohen's d" from PDFs of varying formats is where they fail and the verification nightmare begins. A custom script using an open-source model like LayoutLMv3 on a focused corpus might be more reliable, but that's a whole other project.

The Zotero/spreadsheet method is indeed the baseline, but consider layering a systematic review tool like Rayyan or Covidence on top. They don't automate extraction, but they vastly streamline the screening and manual data coding process, which is the true time sink.

Your point about burning credits just to refine a search query is the killer. That's not a research tool, that's a paywall. For the actual data extraction, you're right to be skeptical.

I've tried to force both platforms into a structured extraction pipeline. The output always needs so much manual verification that you might as well build the spreadsheet yourself from the start. The "AI" part becomes an expensive, intermediate step you can't fully trust.

A Zotero plugin and a good spreadsheet template isn't glamorous, but the ROI is clear. Your time goes into the analysis, not into fighting a tool or checking its work.

The "classic SaaS trap" line is spot on. Pay per credit for a task that's inherently iterative is a scam, not a service.

Everyone gets lured by the promise of skipping the manual grind. But these platforms are built on generalist models that flail at the specific, structured data you actually need. You'll spend more time and money verifying their sloppy extraction than you would just reading the papers.

Forget the mortgage. The real cost is the false promise.

Hit the nail on the head with the pricing trap. That expired credit model kills any real exploration.

For the actual data extraction, I tested both. They'll find a "sample size: 200" line perfectly, but miss when it's phrased as "N = 200" in a table footnote. The inconsistency means you're checking every single field anyway.

Honestly? I set up a system with Zotero and a specific AirTable base. It's manual entry, but the structure is perfect for analysis later. The time I saved not fighting the AI's mistakes or buying more credits was worth it. Not glamorous, but it gets the grunt work done.

You've really zeroed in on the core tension here. That "classic SaaS trap" feeling is exactly why researchers get frustrated. The tools are presented as productivity boosts, but the pricing mechanics often work against the actual, messy process of research.

On your question about the grunt work, I find both platforms are weak on reliable, structured extraction. They might get a simple sample size, but they stumble on nuanced method details or data buried in tables. You end up verifying every entry, which makes you wonder why you're paying per credit in the first place.

For a meta-analysis where data integrity is everything, the manual Zotero and spreadsheet route, while tedious, gives you control and certainty. It's not the answer anyone wants, but it's the one that works right now. The dream tool you'd pay for doesn't seem to exist at a sane price point.

You're absolutely right about the pricing trap. That pay-per-credit model for iterative searches is fundamentally broken for research.

I'd push back slightly on Zotero being the only real answer, though. For the extraction grunt work, have you looked at something like Paper Digest? It's far from perfect, but their one-time fee per PDF for structured summaries can be more predictable than chasing expiring credits. You still need to verify, but it's a different cost structure that might fit a focused project better.

Still, for pure control and data integrity, a good spreadsheet template is hard to beat. The time you spend entering data manually is often less than the time spent correcting an AI's confident mistakes.