Step-by-step: How we used Claw's API to backfill a year of lost historical data post-move.

Hey everyone! 👋 We just wrapped up a massive data migration after moving from our old, clunky project platform to the shiny new one. The worst part? We realized a year's worth of historical task data, including comments and status changes, got left behind in the old system during the initial cutover. Panic!

Instead of accepting the loss, we decided to use Claw's (our old platform) API to extract everything and rebuild the history in our new system. Here’s how we did it, step-by-step:

**The Problem & The Goal:**
* We had ~10,000 completed tasks from the past year stuck in Claw.
* The data we needed: task name, description, creation date, final status, comments, and the assignee at closure.
* Our new platform had a robust API for creating tasks and adding comments, but we needed to map old user IDs to new ones.

**Our Process:**

1. **API Exploration:** First, we got our hands on Claw's API docs. We built a simple script to pull a single task with all its fields and comments to understand the JSON structure.
2. **The Extraction Script:** We wrote a script that looped through all projects in Claw, fetching completed tasks created within our "lost" date range. We paginated carefully to avoid timeouts. The output was a set of JSON files—one per project—stored locally as our backup.
3. **Data Transformation:** This was the tricky part. We had to:
* Map old user emails to new system user IDs.
* Convert Claw's status names to our new platform's status IDs.
* Format the dates to the ISO string our new system required.
* Structure the comment payloads correctly.
4. **The Load Script:** A separate script read our cleaned JSON. For each task, it would:
* POST to create the task in the new system with the original creation date (this was a specific API flag).
* Then, if the task had comments, it would make subsequent POST calls to add each comment with its original timestamp.

**Key Takeaways:**
* **Rate Limiting:** We had to add delays between API calls to both systems to avoid getting blocked.
* **Idempotency:** We made our script re-runnable by checking if a task with a unique identifier (we used the old task ID in a custom field) already existed.
* **Validation:** We ran the script on a small subset (one project) first and had the team verify the recreated tasks looked correct before the full backfill.
* **Team Buy-in:** We showed the team a demo of the "resurrected" project history, which got everyone excited and on board with the process.

The result? We successfully backfilled all 10k tasks with their full comment threads. It felt like magic to the team, and now we have a complete historical record. The scripts are now safely documented in our team's playbook for any future migrations!

If you're facing a similar data rescue mission, I'm happy to share more details on the transformation logic or the script structure. It was a weekend project that saved us from a huge historical black hole.