I've been testing Elicit against a stack of recent law review PDFs. The core promise is solid, but accuracy for *legal citations* is hit-or-miss.
My main gotchas:
* **Pinpoint page numbers are often wrong.** E.g., it'll cite a footnote but give the page the footnote *references*, not where it actually is.
* **Statute and case law citations get mangled.** It might correctly identify "410 U.S. 113" but then link it to a completely unrelated paper in the results.
* It's great for finding **semantic concepts**, but for **precise legal Bluebook-style citation extraction**, you still need to manually verify everything.
My workflow: dump a PDF in, use Elicit's "cited by" to find related papers fast, but then I cross-check every single citation with the original PDF before I'd ever use it in a draft. The "Summary of cited by" feature is more reliable for getting the gist than the raw citation data.
Has anyone found a reliable setup or tool chain that actually nails legal citation extraction? Or is manual verification still the only real answer?
// chris
metrics not myths
I ran into similar issues with pinpoint pages. Have you tried Juris-M/Zotero? It's built for legal citation and can handle the Bluebook style directly. The learning curve is a bit steep, but it parses statutes and cases from PDFs more accurately in my limited testing.
You've nailed the core problem - these tools are built for academic papers, not the weirdly specific formatting of legal citations. Elicit's AI is looking for semantic patterns, not Bluebook rule 10.
I had to deal with something similar automating case law reference pulls for an internal wiki. We ended up with a janky two-step pipeline:
1. A regex monster (seriously, it's ugly) to scrape the raw citation text from the PDF.
2. A separate script that just validates the format against a Bluebook rule subset and pings a legal API for the actual case info.
It works about 85% of the time. The other 15%? Manual review, just like you said. Sometimes the footnote placement or parallel citations break the parser.
Maybe the real answer is a dedicated legal-tech tool, not an adapted academic one. The Juris-M suggestion above might be closer to the mark.
NightOps