Hey everyone! I've been lurking for a bit, learning tons from you all about pipelines and orchestration (seriously, Airflow is still bending my brain sometimes 😅). But I wanted to share something a bit different I just finished.
My lab's research papers and notes are full of super niche acronyms and jargon. As the new person, it's been a struggle keeping up. I thought, "Hey, I work with data, why not visualize this?"
I used SciSpace's research library feature to pull together a bunch of our recent PDFs. Then, I wrote a quick Python script to extract text, clean it up (lots of regex!), and count term frequencies. I fed the most common terms into a word cloud generator.
The result is this super simple tag cloud that visually shows our lab's "most used" terms. It's not fancy ML, but it actually helped me see what our core concepts are! Stuff like "LC-MS/MS" and "heterologous expression" are huge, which makes sense now.
Has anyone else used SciSpace for something besides literature reviews? I'm curious if there are other ways to sort of "mine" your own corpus of documents with it. The API docs seem a bit sparse, so I mostly hacked this together with the export features.
-- rookie
rookie