I've been using Perplexity for a few months to quickly grasp new papers on stream processing and data mesh architectures. I generally like it, but a colleague swears by Elicit for "actual research work." So, I decided to run a small, concrete test.
I fed the same recent paper—"Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics"—into both tools with the same prompt: "Summarize the key architectural principles and the primary criticisms mentioned in this paper." Here's what stood out:
**Perplexity:**
* The summary was more fluid and readable, like a blog post digest.
* It correctly identified core principles like transaction support, schema enforcement, and open formats.
* However, it was weaker on the "criticisms" part, glossing over some nuanced trade-offs about metadata layer complexity that the paper explicitly discusses.
**Elicit:**
* The output was more structured, almost like bullet points extracted directly from the text.
* It pulled specific sentences about limitations verbatim, which felt more "accurate."
* The tone was dryer, and it missed a bit of the high-level synthesis—the "so what" for an engineer.
My takeaway? Perplexity feels better for a quick, conversational understanding, especially when you're exploring a new field. Elicit seems more rigorous for extracting specific claims or data points, maybe for a literature review.
But I'm curious about others' experiences. Has anyone done a more systematic accuracy check, maybe with a quantitative method? I'm thinking about precision/recall on specific claims, but that's a lot of manual labeling. There's a data pipeline in there somewhere...
—Claire