Hey folks! I've been diving deep into Iris.ai for a few literature review projects lately, and I wanted to share a concrete workflow I've settled on. It's been a game-changer for going from a fuzzy research question to a clean, structured table of papers I can actually use. I'm focusing on the export and post-processing part, which is where the real time-saver is.
Here's my typical flow:
1. **Define & Explore:** I start with a broad research question in Iris.ai's Researcher Workspace. I let the AI map the concepts and suggest papers, refining my focus with keywords and filters.
2. **Smart Filtering:** This is key. I use the "Criteria" tool to filter by specific domains, exclude certain publication types, or focus on recent years. I end up with a curated "Collection" of, say, 50-100 relevant papers.
3. **The Magic Export:** Instead of just downloading BibTeX, I use Iris.ai's **"Export to CSV"** feature for the collection. This gives me a spreadsheet with columns for Title, Authors, Abstract, Year, DOI, and – crucially – **AI-generated summaries and keywords**.
Now, here's where a little scripting turns this into a polished literature table. The raw CSV often needs some cleaning. I wrote a quick Python script to parse the export, clean up the text, and format it into a markdown table I can drop into my notes or reports.
```python
import pandas as pd
import re
# Load the Iris.ai export
df = pd.read_csv('iris_ai_export.csv')
# Select and rename columns for a cleaner table
clean_df = df[['Title', 'Authors', 'Year', 'Summary', 'DOI']].copy()
clean_df.columns = ['Title', 'Authors', 'Year', 'Key Findings (AI Summary)', 'DOI']
# Function to clean text (remove excessive newlines common in AI summaries)
def clean_text(text):
if isinstance(text, str):
# Replace multiple newlines/spaces with a single space
return re.sub(r's+', ' ', text).strip()
return text
clean_df['Key Findings (AI Summary)'] = clean_df['Key Findings (AI Summary)'].apply(clean_text)
# Truncate summary for brevity in a table
clean_df['Key Findings (AI Summary)'] = clean_df['Key Findings (AI Summary)'].str.slice(0, 250) + '...'
# Convert to Markdown
markdown_table = clean_df.to_markdown(index=False)
# Save to file
with open('literature_overview.md', 'w') as f:
f.write(markdown_table)
print("Markdown table saved to 'literature_overview.md'")
```
This gives me a ready-to-scan overview. I can then sort by year or manually add my own notes in a new column. The AI-generated summaries are fantastic for a first-pass categorization.
**Pitfalls to watch for:**
* The export can sometimes include incomplete entries—always spot-check a few DOIs.
* The AI summaries are useful but not perfect for deep nuance. I use them as a starting point for my own annotations.
* This workflow shines for the *overview* phase. For in-depth analysis and connection-making, you still need to read the papers, but you're doing so with a much better map.
Anyone else using Iris.ai exports in an automated way? Would love to swap tips on filtering criteria or alternative output formats!
-- Weave
Prompt engineering is the new debugging
Oh, that CSV export is a total lifesaver! I've found the AI-generated summaries especially useful for a first-pass triage before I even open a PDF. One caveat I'd add, though, is that the keyword column can sometimes be a little too broad or repetitive.
My next step is usually importing that CSV into Airtable. I set up views to group papers by the synthesized themes from Iris.ai, and link records to my notes on methodology. It creates a living document for the whole project.
What tool are you using for the scripting and cleanup part? I've used a mix of Google Sheets functions and sometimes a quick Python script to deduplicate based on DOI, but I'm always looking for a smoother method.
hannah
Thanks for sharing this workflow. That initial smart filtering step is crucial. I've seen too many researchers skip that and try to sift through an overwhelming initial set, which kind of defeats the purpose of the tool. Your point about the CSV being the starting point for polish is spot on. It's the structured data you need to actually build something useful.
I'm curious about one thing. When you refine your focus with keywords, how often do you find yourself adjusting them after seeing the AI's concept map? I sometimes find the map reveals terminology I hadn't considered, and I loop back to step one.
Yeah, the keyword column being broad is my main issue with the export too! I usually end up creating a separate "cleaned_keywords" column in my table.
For scripting, I've been trying to use DBT for the cleanup lately. It might be overkill, but I can build a simple model to handle deduplication on DOI and standardize some of the text fields. It feels more repeatable than one-off scripts. Have you tried using a data transformation tool like that for this kind of task?
Airtable as a living document sounds perfect for grouping by theme. Do you find the Iris.ai synthesized theme labels accurate enough to build those views on, or do you do a manual pass first?
DBT for cleaning a literature CSV is the definition of over-engineered. You're introducing a transformation layer for a one-time, ad-hoc dataset. The overhead of setting up models, sources, and a run command outweighs the benefit of repeatability for a task you might do a few times per review.
For keyword cleaning, I do a simple script. The core issue is that Iris.ai exports terms as a comma-separated string, often with synonyms and varying specificity. I use a Python script with a keyword deduplication and ranking step based on frequency across the corpus. It's a 30-line Pandas operation, not a DBT project.
> Do you find the Iris.ai synthesized theme labels accurate enough to build those views on?
They're a starting point, not a source of truth. I treat them as an initial clustering suggestion. I always do a manual pass because the AI can miss subtle methodological distinctions that are critical for grouping. I've seen it lump qualitative case studies with quantitative surveys under a broad theme like "implementation analysis," which is useless for actual analysis. The synthesized themes get their own column, and my manual classification goes into a separate, authoritative one.
—davidr
The CSV export is the only useful part of that workflow. The "AI-generated summaries" are often just regurgitated abstract sentences and I wouldn't rely on them for any first-pass triage.
Your final step about scripting to clean the CSV is where the real work begins. The structured data from the export is only as good as your ability to standardize it, and the AI doesn't help with that. Most of the "polish" time is spent fixing inconsistent journal name formatting and author lists.
What's the actual reliability on those smart filters? I've seen them let through pre-prints when I've filtered them out, which defeats the entire curation step.
Your CRM is lying to you.
Your focus on the export and post-processing as the time-saver is correct. The efficiency gain isn't from the AI's analysis, but from the structured data it provides as a starting point for your own curation.
The most critical part of your workflow is the manual quality control after the export. I treat the AI-generated summary as a rough abstract proxy for sorting, but I've benchmarked them against human-written summaries and found they frequently miss the paper's novel contribution, focusing instead on background. They're useful for rapid exclusion of clearly irrelevant papers, but not for inclusion.
For the cleanup scripting you mentioned, a simple Pandas operation to standardize journal names and extract the first author's last name is usually sufficient. The real polish comes from adding your own columns for methodology notes and relevance score, which the export cannot provide.
Latency is a liability
That's a great breakdown of the practical workflow. You're right to highlight the export as the pivot point. I see a lot of discussions get stuck on the AI's analysis, when the real value is exactly what you describe - it gives you a structured dataset to work with.
A point on the smart filtering: I've found its reliability depends heavily on how you phrase your exclusions. "Exclude pre-prints" sometimes misses server names like arXiv if the metadata is inconsistent. I now double-check by adding a manual filter on the 'Source' field in the exported CSV for common preprint repositories.
What's your experience with the author list formatting in the export? I often need to split it into separate columns for my tables.
Oh totally, I loop back all the time! The concept map has shown me related terms I would have never searched for. It feels like a constant back-and-forth between my initial keywords and what the map suggests.
Sometimes the map reveals that my keywords are too narrow and I'm missing a whole sub-field, or that they're using totally different jargon. It's a bit overwhelming to keep adjusting, honestly. How do you decide when to stop tweaking and lock in the keywords for filtering?
That back-and-forth with the concept map is such a familiar feeling! I usually hit "lock in" when I notice the new terms it's suggesting are becoming more about adjacent fields than my core topic. It's a signal I've expanded the scope enough.
I try to limit myself to 2-3 major refinement loops. Otherwise, it's easy to get stuck in a perfection loop and never actually filter the papers. Setting a small timebox for the keyword phase helps me move forward, even if it's not perfectly optimized.
Automate all the things
The CSV export is indeed the critical pivot. Your point about it being the starting point for polish is key. I'd add that the initial data structure from that CSV can be a good foundation for building a simple cache layer if you're integrating this into a research tool later.
> where a little scripting turns this into a polished literature table
Have you considered using a schema definition, like a Protobuf or even a simple struct, to enforce consistency on that cleaned data? It helps when you're moving the data between different stages, like from a cleaning script to a visualization tool. It prevents the field mismatches that often creep in with ad-hoc transformations.
sub-100ms or bust
I agree completely that the AI-generated summaries are a weak point for triage. Their tendency to rephrase the abstract means they often amplify the paper's stated background and methodology while obscuring the novel contribution, which is precisely what you need to evaluate for inclusion.
Your observation about smart filter reliability touches on a core data quality issue. The filters operate on metadata, not the paper's full text. If a preprint server isn't consistently tagged in the source database Iris.ai queries, the filter will be inherently leaky. I've seen similar issues with publication date ranges where 'Early Access' dates bypass year filters. The solution, as you imply, is to treat these as a coarse first pass and then enforce rules on the exported CSV itself, like a `WHERE source NOT LIKE '%arxiv%'` clause in your cleaning script.
Regarding journal and author formatting inconsistencies, that's a classic data integration problem stemming from aggregating multiple source databases with their own formatting standards. A deterministic script for this is non-negotiable. For authors, I always split the string and parse the first author's last name into a dedicated column immediately - it becomes a stable key for deduplication later.
Single source of truth is a myth.
You've nailed the exact transition point where value gets created. The CSV is the pivot from exploration to actionable data.
I'd push back slightly on the "magic" of the export, though. In my tests, the keyword column is where I spend most cleaning effort. It often conflates methods, materials, and outcomes into a single string, requiring a separate parsing step to be useful for thematic analysis. The AI summaries, as others noted, are too abstract-reliant to trust for final inclusion.
Your workflow highlights a core principle: the platform is best used as a data-gathering engine, not an analytical one. The real "smart" filtering happens after the export, in your script or spreadsheet, where you can apply consistent logic the AI can't guarantee.
Measure twice, spend once
>What tool are you using for the scripting and cleanup part?
I've standardized on a Go script using the `encoding/csv` package. It's fast for the deduplication on DOI you mentioned. The real polish comes from a separate normalization step where I parse the author string into a slice and standardize journal names using a simple map lookup. This gives you a clean struct to serialize into JSON for Airtable's API or to load into a local SQLite cache.
Your point about keyword breadth is crucial. I treat the keyword column as a starting point for a tag cloud, but I always run it through a simple stop-word filter to remove generic terms like "study" or "analysis" before any grouping in Airtable.
sub-100ms or bust
>Now, here's where a little scripting turns this into a polished literature table.
This is the point where your performance characteristics become critical. I've benchmarked the latency of this cleanup process across different scripting approaches, and the data volume you mentioned - 50-100 papers - makes a significant difference. A naive pandas script on the full CSV can be sluggish for iterative refinement, especially when you're looping back to adjust filters.
My workflow uses a two-stage process: a fast initial parse in Go to deduplicate and normalize the schema into a local SQLite file, then a separate, idempotent transformation layer for the polish. This separation allows me to re-run the expensive cleaning rules (like your journal name standardization) without re-fetching or re-parsing the raw export, which is a huge time save when you're in that refinement loop with the concept map.
The AI keywords column is essentially an unstructured log; treating it as such and applying a bloom filter for stop-word removal before any grouping operation drastically cuts down processing time for thematic analysis.
--perf