As a team deeply invested in Site Reliability Engineering practices and operational efficiency, we have long struggled with the qualitative feedback loop following post-incident reviews and project retrospectives. While our technical monitoring via Prometheus and Grafana provides quantitative data, capturing actionable, consistent feedback from meeting participants remained a manual, inconsistent process. This led to fragmented insights and lost opportunities for process improvement.
We decided to leverage Fellow's API to automate the collection and structuring of this feedback, integrating it directly into our observability and documentation workflows. The primary goals were:
* To ensure 100% consistency in soliciting feedback after every designated meeting.
* To structure unstructured narrative feedback into quantifiable metrics for trend analysis.
* To reduce the overhead for meeting facilitators and increase participant response rates.
* To create a bidirectional flow where meeting insights could trigger changes in our technical monitoring alerts.
The implementation required a systematic approach, broken down into the following phases.
**Phase 1: API Authentication and Meeting Identification**
Fellow's API uses OAuth 2.0. We created a dedicated service account for our automation bot. The first step was to fetch meetings for a given timeframe and filter for those tagged as "Post-Mortem" or "Retrospective" using the `GET /meetings` endpoint. We key on the `ended_at` timestamp to trigger the feedback workflow.
```bash
# Example curl to list recent ended meetings
curl -X GET "https://api.fellow.app/v1/meetings?ended_after=2024-01-15T00:00:00Z"
-H "Authorization: Bearer $FELLOW_ACCESS_TOKEN"
-H "Content-Type: application/json"
```
**Phase 2: Automated Feedback Request Generation**
Upon identifying a completed meeting, the system automatically creates a feedback request for every participant. This is done via the `POST /feedback/requests` endpoint. We template the request message to include specific, actionable questions aligned with our SRE pillars (e.g., "How effective was the incident timeline reconstruction?").
**Phase 3: Structured Data Aggregation and Storage**
As feedback is submitted, we poll for responses using `GET /feedback/requests/{id}`. We parse the JSON responses and apply a simple sentiment/keyword scoring model (using a pre-defined dictionary of positive/negative operational terms like "clear," "delayed," "ambiguous," "actionable"). The aggregated data is then written to a timeseries database (VictoriaMetrics, compatible with Prometheus) alongside the meeting's metadata.
```yaml
# Sample metric emitted to our monitoring system
fellow_feedback_score_summary{meeting_id="abc123", meeting_type="post_mortem"} 4.2
fellow_feedback_comment_count{meeting_id="abc123", sentiment="positive"} 7
fellow_feedback_comment_count{meeting_id="abc123", sentiment="negative"} 3
```
**Phase 4: Integration with Observability and Alerting**
This is where the loop closes. We added a Grafana dashboard panel tracking the `fellow_feedback_score_summary` alongside relevant technical KPIs from the same incident (e.g., MTTR, alert volume). A consistent dip in feedback scores triggers a low-severity alert for our engineering manager, prompting a qualitative review. Conversely, when technical metrics degrade, we can automatically schedule a Fellow meeting for analysis, creating the ticket directly via the `POST /meetings` API.
**Key Takeaways and Pitfalls:**
* The API rate limits are generous but must be respected; implement exponential backoff in your client.
* Participant email addresses must match their Fellow account exactly for the feedback request to be routed correctly—we had a ~15% mismatch initially due to internal alias usage.
* Storing the raw, anonymized comments in a data lake (S3) has proven valuable for later longitudinal analysis using NLP techniques, beyond the simple scoring model.
* The automation has increased our feedback response rate from an estimated 40% to 98%, providing a significantly richer dataset for continuous improvement.
This integration has effectively bridged a gap between our quantitative observability stack and qualitative human processes, making our operational reviews more data-driven and actionable. I am interested in hearing how others may have structured their feedback scoring models or integrated similar tools with Jira or Linear.