Hey folks, mid-migration here and I just had a minor panic moment. We're moving from a legacy CRM to a new platform, and during the final cutover data sync, a few thousand records failed to map properly due to some unexpected field constraints. The sync job itself logged the errors... but to a file, hours after the fact.
We needed a way to see these failures *as they happened* so we could triage and potentially pause the migration. I couldn't find a simple, ready-made tool that fit our stack, so I spent an afternoon building a dead-simple webhook listener with FastAPI to catch and alert on sync failures in real-time.
The new CRM allows us to configure webhooks for "sync error" events. I stood up a tiny service that listens for those POSTs, parses the error details, and immediately pushes them to a Slack channel. It's been a lifesaver for monitoring the health of the migration without staring at logs.
Here's the core of it (heavily simplified, but you get the idea):
```python
from fastapi import FastAPI, Request, HTTPException
import httpx
from pydantic import BaseModel
import os
SLACK_WEBHOOK_URL = os.getenv("SLACK_WEBHOOK_URL")
app = FastAPI()
class SyncErrorEvent(BaseModel):
record_id: str
error_code: str
message: str
object_type: str
@app.post("/webhook/crm-sync-error")
async def handle_sync_error(request: Request):
# In reality, you'd want proper signature verification here!
payload = await request.json()
# Validate and extract the key bits
try:
error_event = SyncErrorEvent(**payload["data"])
except Exception:
raise HTTPException(status_code=400, detail="Bad payload")
# Craft a Slack message
slack_msg = {
"text": f"🚨 CRM Sync Error",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": f"*{error_event.object_type}* record `{error_event.record_id}` failed with error `{error_event.error_code}`:n>{error_event.message}"
}
}
]
}
# Async call to Slack
async with httpx.AsyncClient() as client:
await client.post(SLACK_WEBHOOK_URL, json=slack_msg)
return {"status": "ok"}
```
I containerized it, threw it on a small instance, and added some retry logic. The peace of mind during the actual cutover was immense. We caught a recurring issue with phone number formatting within minutes instead of hours.
If you're planning a migration, I *highly* recommend building or setting up a real-time feedback loop for errors. The default logging/reporting from most migration tools often isn't fast enough when you're in the thick of it.
Has anyone else built similar ad-hoc monitoring for their migrations? Curious about other approaches.
~d