Step-by-step: How I use custom columns to extract population details

Alright, let's get this over with. Another "AI-powered literature review" tool, but this one at least lets you get your hands dirty with the data. Elicit's main draw is the automated extraction, but anyone who's been around the block knows the automated part is only as good as your ability to shape the output. The real work is in the post-processing. That's where the custom columns feature stops being a shiny button and starts being a useful lever.

I was tasked with pulling a consistent set of population details (size, demographics, location) from about 200 papers on urban development. The default extraction was, predictably, a mess—sometimes it grabbed sample size, sometimes it grabbed city population, sometimes it just gave me a paragraph. Here's how I wrangled it into a usable CSV without spending a week in Excel hell.

First, you have to think like a parser. Don't ask for "population details." Be excruciatingly specific. I set up three separate custom columns, each with a prompt that leaves minimal room for interpretation. The key is to ask for a single, structured piece of data per column.

**Column 1: `study_population_size`**
* **Prompt:** "Extract the exact numerical sample size (N) of the primary study population as a single integer. If multiple numbers are given, use the one explicitly labeled as the final analysis sample. Respond only with the number or 'Not specified'."
* **Result:** This cuts through all the "initial recruitment was X, but after attrition..." nonsense and gives you a clean integer or a clear null.

**Column 2: `population_location`**
* **Prompt:** "State the primary geographic location of the studied population (e.g., 'Chicago, Illinois, USA', 'National survey, Japan'). If multiple locations, list only the first major one. Use the format 'City, State/Region, Country'."
* **Result:** Forces a standard format, making later geocoding or grouping possible.

**Column 3: `demographic_summary`**
* **Prompt:** "Summarize key demographics in a pipe-separated list. Format: 'mean_age=X|female_percent=Y|key_criteria=Z'. Example: 'mean_age=45.2|female_percent=52|key_criteria=diagnosed with COPD'."
* **Result:** This is the hack. Elicit will often spit out a messy sentence, but asking for a specific delimiter format means you can later split this in your data warehouse or even in Excel using text-to-columns.

Once you run the extraction, you export to CSV. The real pipeline starts there. I throw the CSV into a Postgres staging table using `copy`, then run a series of `UPDATE` statements to clean the extracted strings. For example, to parse that `demographic_summary` column:

```sql
-- Create separate columns from the pipe-delimited field
ALTER TABLE elicit_staging
ADD COLUMN mean_age FLOAT,
ADD COLUMN female_percent FLOAT,
ADD COLUMN key_criteria TEXT;

-- Update using split_part, with error handling
UPDATE elicit_staging
SET
mean_age = NULLIF(split_part(demographic_summary, '|', 1), '')::REPLACE('mean_age=', '', '')::FLOAT,
female_percent = NULLIF(split_part(demographic_summary, '|', 2), '')::REPLACE('female_percent=', '', '')::FLOAT,
key_criteria = NULLIF(split_part(demographic_summary, '|', 3), '')::REPLACE('key_criteria=', '', '')
WHERE demographic_summary LIKE '%|%';
```

Is it elegant? No. It's duct tape and string. But it turns Elicit from a fancy PDF highlighter into a semi-reliable data ingestion node. The lesson, as always: the tool doesn't do the work for you. It just gives you a slightly better lump of clay to sculpt. The moment you treat its output as final, you're doomed. You need a pipeline, even if it's just a few SQL statements, to enforce schema and quality.

-- old salt