Hey team! We've been automating our entire gitops pipeline, but our sprint retrospectives are still super manual. We gather Slack threads and meeting notes, then someone has to manually populate the "What went well"/"What didn't" columns in Notion. Seems like a perfect job for an AI workflow!
Has anyone built a pipeline for this? I'm thinking:
1. Collect raw text from Slack/Google Meet transcript/Zoom comments.
2. Use AI (like Notion AI API or maybe OpenAI?) to categorize each comment as "went well", "needs improvement", or "action item".
3. Push the structured data into a Notion database (our retrospective board).
I'd probably orchestrate this with a GitHub Action on a schedule. Something like:
```yaml
name: Populate Retrospective
on:
schedule:
- cron: '0 18 * * 5' # Friday after sprint
jobs:
process:
runs-on: ubuntu-latest
steps:
- name: Fetch comments
uses: actions/github-script@v6
# ... fetch from sources
- name: Categorize with AI
run: |
# Call LLM API, get structured JSON
- name: Update Notion Board
uses: shuya7/notion-action@v1
```
Curious if anyone has tackled this already? How do you handle the prompt engineering to get consistent categorization? And what about secrets management for the API keys? 😅
> git commit -m 'done'
git push and pray
That's a solid approach, and automating the data collection part is definitely doable with scripts. One thing I'd consider is the nuance of categorization. A comment like "We finally deployed the feature" could be a 'went well' or, if it was late and painful, an 'action item' about planning. The AI might need more context or a confidence threshold, otherwise you risk misplacing team sentiment.
You might want to add a lightweight human review step before the data hits the main board, like a staging area. It catches mis-categorizations and keeps the team's voice authentic. Maybe run it Thursday evening so someone can quickly validate Friday morning? Just a thought.
Also, watch out for privacy - make sure you're only pulling from public team channels or with consent. Would hate to see a private vent end up on the board 😅
Keep it constructive.
We've actually done something similar using the OpenAI API directly. Your GitHub Action approach is good for scheduling, but I found it easier to run as a small FastAPI app that triggers from a webhook when the retro channel gets archived.
Key thing we learned: the AI prompt is everything. We feed it the raw text plus a few examples of each category from past retros (like 2-3 per type) for few-shot learning. Accuracy shot up compared to just asking it to categorize.
One caveat - Notion's API rate limits can bite if you're creating dozens of rows at once. Add a small delay between calls in your final step.
Automate the boring stuff.
That few-shot learning trick is a great idea - hadn't thought to prime the model with actual past retro items. How do you handle it when a single comment contains multiple sentiments? Like "Deployment was smooth but the docs were outdated" - do you split it into two entries, or let the AI pick the dominant one?
Also, webhook trigger from archiving the channel is clever. Means it runs exactly when the raw material is 'final'.
Great question about mixed sentiment comments! We ran into this too. Our approach is to prompt the AI to split them - so that "Deployment was smooth but the docs were outdated" becomes two separate entries, one positive and one action item. It keeps the board clean and actionable.
The webhook trigger is fantastic for timing, though we had to add a quick check to make sure all team members had posted their thoughts before the channel auto-archived. Otherwise, you might process an incomplete dataset. Maybe a simple "ready to process?" poll in the channel first?
keep building