I use Elicit to find and extract data from ML papers, then pipe it into Airtable for tracking and analysis. The goal is a continuously updated "living review" of model architectures and benchmarks.
Workflow:
1. **Elicit Search & Extraction:** I run a focused query (e.g., "vision transformer efficiency 2023"). Elicit returns papers and, crucially, lets me extract key fields into a CSV.
2. **Automated Ingestion:** A simple Python script transforms the CSV and pushes to Airtable's API. Key extracted fields:
* Model name
* Reported accuracy (Top-1, etc.)
* Throughput (samples/sec)
* Hardware platform
* Dataset
```python
import csv
from airtable import Airtable
# ... read Elicit CSV, clean data ...
airtable = Airtable('base_key', 'table_name', api_key='key')
for row in cleaned_data:
airtable.insert(row)
```
3. **Airtable as Living Dashboard:** The Airtable base becomes the single source of truth. I use:
* Grid view for raw data.
* Gallery view with linked PDFs.
* Gantt chart view to track publication timelines.
* Formula fields to compute derived metrics (e.g., accuracy/throughput ratio).
The main benefit is traceability. Every performance figure is linked directly to its source PDF. I can filter and sort across hundreds of papers in seconds, something a static PDF review can't do. The pipeline runs weekly, keeping the dataset current.
Numbers don't lie.
That's really cool, thanks for sharing! I'm still getting my head around Python for automation. How do you handle it when the CSV structure from Elicit changes? Do you just update your script manually, or is there a trick to make it more flexible?
Great question. That's the part of the script I update most often, honestly. I've found the CSV columns from Elicit are pretty stable for core things like title and authors, but the custom extractions you set up can shift a bit, or they'll add a new default column after an update.
My trick is to have the script log the column headers it sees every single run to a simple text file. That way, if an import fails, my first check is that log to see if the structure changed overnight. Then it's a quick manual tweak to map the new field names. It's not fully automated, but the logging gives me an early warning so the whole pipeline doesn't just break silently.