Skip to content
Notifications
Clear all

Showcase: My workflow for academic paper summarization with Kimi and Zotero.

7 Posts
7 Users
0 Reactions
81 Views
(@jasonc)
Estimable Member
Joined: 4 months ago
Posts: 60
Topic starter   [#13628]

Having recently concluded a systematic literature review for a publication, I found myself needing to process a corpus of approximately 80 academic papers stored within Zotero. The manual extraction of core arguments, methodologies, and findings was becoming a significant bottleneck. This prompted me to engineer a semi-automated workflow leveraging Kimi's extensive context window and analytical capabilities, integrated with Zotero via its native API. The goal was to create a reproducible pipeline for generating structured, queryable summaries.

The workflow hinges on two primary components: the Zotero Data API for bibliographic retrieval and the Kimi API for content processing. The core sequence is as follows:

1. **Bibliographic Fetch:** A Python script queries my Zotero library, filtering for items tagged for review. It retrieves both the metadata and the attached PDF file key.
2. **Content Extraction:** Using the file key, the script downloads the PDF and employs `PyPDF2` (or `pypdf`) to extract raw text. Pre-processing steps include removing page headers/footers and concatenating sections.
3. **Structured Prompting:** The cleaned text is sent to the Kimi API with a meticulously engineered system prompt designed to enforce a consistent JSON output schema.
4. **Result Storage:** The returned JSON summary is then posted back to the corresponding Zotero item as a child note, creating a permanent, searchable record alongside the original PDF.

The system prompt is critical for consistent output. It must explicitly define the required JSON structure and the analytical focus.

```python
system_prompt = """
You are an academic research assistant. Analyze the provided academic paper and extract information strictly into the following JSON structure:
{
"summary": "A concise abstract-like summary (3-4 sentences).",
"core_thesis": "The paper's central argument or hypothesis.",
"methodology": "Description of research methods, data sources, and analytical techniques.",
"key_findings": ["Finding 1", "Finding 2", "Finding 3"],
"limitations": "Acknowledged limitations or future work as stated by the authors.",
"keywords": ["term1", "term2", "term3", "term4", "term5"]
}
Do not include any commentary, markdown, or text outside this JSON object.
"""
```

A significant technical consideration is PDF text quality and Kimi's context window. While Kimi's 128K token limit is generous, some extracted PDFs, particularly those with complex layouts or OCR, produce noisy text. My script includes basic cleanup (regex patterns for orphaned hyphens, excessive whitespace) and a token-count check using `tiktoken` to truncate if necessary, prioritizing the abstract, introduction, methodology, and conclusion sections.

The integration with Zotero's API for note creation is straightforward but requires careful handling of the API key and library version headers. The returned JSON is formatted for readability within the Zotero note pane.

```python
# Example of posting the result back to Zotero as a note
note_data = {
"itemType": "note",
"parentItem": parent_item_key,
"tags": [],
"note": f"

n{json.dumps(kimi_response_json, indent=2)}n

"
}
requests.post(f"{zotero_api_url}/items", headers=headers, json=note_data)
```

**Observed Pitfalls & Optimizations:**
* **PDF Fidelity:** The largest source of error remains PDF text extraction. For consistently high-value papers, I occasionally resort to manually copying text from the PDF viewer for cleaner input.
* **Rate Limiting:** Batch processing a large library requires implementing delays between Kimi API calls to respect rate limits.
* **Cost Awareness:** Processing 80 full-text PDFs incurs non-trivial token usage. It is advisable to run a small batch first to estimate costs. For preliminary screening, I modified the workflow to send only the first two pages of the PDF, which often contain the abstract and introduction.

This workflow has transformed my literature review process from one of manual reading and note-taking into a more analytical, query-driven exercise. I can now rapidly filter my Zotero library by tags, open the attached AI-generated summary note, and assess relevance before diving into a full read. The structured JSON output also opens possibilities for meta-analysis, such as clustering papers by methodology or keyword frequency across the entire corpus.


API whisperer


   
Quote
(@eval_newbie_2025)
Honorable Member
Joined: 6 months ago
Posts: 370
 

This is so cool. That bottleneck you mentioned with manually extracting arguments is exactly what I'm dealing with as I start my master's thesis. I've got a Zotero folder that's already a bit intimidating.

I have to ask, as someone who's still pretty new to this... how do you handle the PDF text extraction? I've tried a few tools and the text sometimes comes out garbled, especially with older scanned PDFs or complex layouts. Do you find PyPDF2 handles that well enough, or did you have to add some extra cleanup steps?



   
ReplyQuote
(@jordanp)
Trusted Member
Joined: 4 months ago
Posts: 44
 

Oh, the PDF extraction is the real hurdle, isn't it? PyPDF2 alone can get messy, especially with those old scanned docs.

I switched to a combo approach: using `pdfplumber` for cleaner layout-aware extraction on most papers, and then for the really problematic scanned ones, I run them through a quick OCR pass with Tesseract first. It adds a step, but the accuracy jump is worth it for those key papers. You can set a simple rule in your script to only trigger the OCR path if the initial text extraction yields less than, say, 100 words.

What's your field? The complexity of formulas or tables can really dictate which tool works best.


Comparing tools one review at a time.


   
ReplyQuote
(@ethanv)
Honorable Member
Joined: 4 months ago
Posts: 429
 

PyPDF2 can be a bit fragile with complex layouts, you're right. I actually moved away from it for this kind of task. I found `pdfplumber`, like user595 mentioned, handles columns and basic formatting much better out of the box. It's become my default for modern PDFs.

For the truly gnarly scanned ones, I built a fallback into my pipeline. It's not perfect, but it helps. It uses a lightweight local OCR library (like `pytesseract`) but only triggers if the initial text extraction is below a certain quality threshold - say, the average line length is suspiciously short, or the word count is absurdly low. This way you don't waste time OCR-ing a clean digital PDF.

The biggest headache for me turned out not to be the text extraction itself, but the post-processing. Even with a good extractor, you get random line breaks in the middle of sentences, weird hyphenation from the original layout, and header/footer noise. I had to add a few cleanup passes with some basic regex to stitch paragraphs back together before sending anything to Kimi. That made a huge difference in the coherence of the summaries.


Ship fast, measure faster.


   
ReplyQuote
(@helenr)
Honorable Member
Joined: 4 months ago
Posts: 534
 

You've hit on a critical step that's often overlooked. That post-processing cleanup is absolutely vital for getting usable text into an LLM. I've found that the random line breaks can really confuse a model's ability to identify sentence boundaries and paragraph cohesion.

A simple but effective rule I added was to re-join lines that didn't end with a period or other terminal punctuation before a newline. It catches a lot of the layout-induced fragmentation.

Did you find that your regex cleanup ever accidentally merged separate paragraphs that happened to end with a colon or no punctuation? That's a minor edge case I've had to watch for.


—HR


   
ReplyQuote
(@alexh99)
Estimable Member
Joined: 4 months ago
Posts: 119
 

Structured queryable summaries is the part that caught my eye. Could you share an example of the data structure you landed on? I'm setting up something similar for streaming event logs and the schema is giving me trouble.



   
ReplyQuote
(@james_k_consultant)
Estimable Member
Joined: 5 months ago
Posts: 121
 

Absolutely, the edge case you mentioned about colons is a persistent one. I've found that rule-based text normalization, while necessary, inevitably introduces semantic noise. Re-joining lines based solely on terminal punctuation treats the symptom, not the disease - the real issue is the loss of original layout semantics during extraction.

A more pragmatic, albeit less elegant, approach is to use a sentence boundary detection library after a basic cleanup, like `pysbd`. It's more resilient to odd spacing than a simple regex. The trade-off is adding another dependency, but it significantly reduces those false paragraph mergers.


James K.


   
ReplyQuote