Hey everyone! 👋 I've seen a few threads pop up asking about bringing old Google Analytics data into Fathom, and let me tell you, I just went through this process myself. It was... a journey! I wanted to share my step-by-step experience, including the gotchas and the "aha!" moments, in hopes it saves someone else a weekend of headaches.
My scenario: I had about 18 months of historical data in a Universal Analytics (UA) property that I needed to migrate for year-over-year reporting before we fully sunsetted the old GA view. Fathom's API is fantastic for current data, but historical import requires a custom ETL process. Here's how I structured it:
**The High-Level Plan:**
1. Extract raw session data from Google Analytics Core Reporting API.
2. Transform the data into a format that mimics Fathom's event structure.
3. Load it into Fathom via their Events API.
**Step 1: The Extraction (This was the trickiest part)**
You'll need to use the GA API with a service account. The key here is to batch your requests by month to avoid sampling and quota issues. I used Python and the `google-analytics-data` library.
```python
# Simplified snippet of the extraction loop
from google.analytics.data_v1beta import BetaAnalyticsDataClient
from google.analytics.data_v1beta.types import DateRange, Dimension, Metric, RunReportRequest
client = BetaAnalyticsDataClient.from_service_account_file("credentials.json")
for month in month_ranges:
request = RunReportRequest(
property=f"properties/{GA_PROPERTY_ID}",
dimensions=[Dimension(name="date"), Dimension(name="pagePath")],
metrics=[Metric(name="sessions"), Metric(name="screenPageViews")],
date_ranges=[DateRange(start_date=month_start, end_date=month_end)],
)
response = client.run_report(request)
# Process rows into a manageable DataFrame or list
```
**Step 2: Transformation & Mapping Pitfalls**
This is where you need to think carefully. Fathom's event model is different from GA's session-based model. You're essentially creating a `pageview` event for each session. I decided to:
* Use the GA `date` and `pagePath` as my core data.
* Generate a dummy `hostname` based on my site's domain (Fathom requires it).
* Create a synthetic, unique `session_id` for each row since GA doesn't expose this at the user level for the API.
* Map the GA `sessions` metric to be the *count* of events. One GA session = one Fathom pageview event.
**The Big Gotcha:** Timestamps! Fathom's API expects timestamps in **UTC ISO 8601 format**. GA data is aggregated by date in your property's timezone. I had to assign a "midday" timestamp to each date in UTC to avoid all events appearing at 00:00:00, which skewed Fathom's hourly reports.
**Step 3: The Load**
Fathom's Events API is straightforward. The critical part is batching your requests and respecting the rate limits. I aimed for batches of 100 events.
```bash
# Example curl for a single event
curl -X POST https://api.usefathom.com/v1/events
-H "Authorization: Bearer $FATHOM_API_KEY"
-H "Content-Type: application/json"
-d '{
"name": "pageview",
"domain": "yourdomain.com",
"path": "/blog/post",
"timestamp": "2023-10-15T12:00:00Z",
"unique_visitor_id": "generated_uuid"
}'
```
**My Lessons Learned:**
* **Data Volume:** This creates a *lot* of events. My 18 months of aggregated GA data turned into millions of individual Fathom events. Be prepared for this to affect your Fathom bill if you're on a tiered plan.
* **Data Fidelity:** This is not a 1:1 replica. You lose user journeys, bounce rates (as GA calculated them), and a lot of the granular user-level data. The value is in preserving high-level traffic trends and page popularity over time.
* **Double-Check Mapping:** Spend time aligning your most important pages/paths. Redirects and URL parameter handling between your old and new site can cause duplication.
Overall, I'm happy I did it. Having that historical line on the graphs is invaluable. But it's a project for those comfortable with data pipelines. If you only need a year of data, maybe just keep the old GA property open in a separate tab. 😅
Would love to hear if anyone else has tried this and how you handled things like campaign data or custom dimensions!
—B
Backup first.
Nice breakdown of the extraction step! Batching by month is such a crucial detail - I made the mistake of trying to pull a whole year at first and the sampling was a mess. Did you run into any issues with the date dimensions for sessions, or did you find a metric that mapped cleanly over to Fathom's structure? Also, curious if you used Zapier or any no-code tools for the transform/load steps or if it was all custom code from there?
Automate all the things
Oh, the date mapping was actually smooth! Fathom's sessions map pretty well to GA's sessions metric. The gotcha was pageviews - had to deduplicate events for single-page sessions. I used a custom Python script for the transform, Zapier felt too rigid for the data massaging needed.
Anyone else try using BigQuery as a middle layer? I've heard that can simplify sampling issues for large datasets.
data over opinions
Interesting point about using BigQuery as a middle layer! For large datasets, that's definitely a smart way to avoid sampling entirely. It does introduce another cost and complexity factor though - you're now managing a BigQuery export job and its billing on top of the rest of the pipeline.
I'd add that for folks without the resources for a custom script, using a tool like Stitch or Fivetran to move the data from GA (or BigQuery) into a data warehouse, and then transforming it there with SQL, can be a nice compromise. It still requires some SQL chops, but it's often more accessible than a full Python ETL.
Trust the data, not the demo.
Ah, the classic "just use a third-party connector and SQL" compromise. It sounds sensible until you're staring at a transformed dataset that's technically correct but conceptually useless for your actual sales pipeline. Stitch and Fivetran are great for moving bits, but they don't help you decide *which* bits to keep.
My caveat, from doing this for revenue data: the transform logic is the whole game. Writing that in SQL can be more opaque and harder to version-control than a simple Python script. You're trading one complexity (code) for another (black-box connector configurations and warehouse-specific SQL dialect). The billing for those tools also scales with row counts, so if your historical GA data is large, the "compromise" gets pricey fast.
I've found it's better to invest the weekend in the script. At least when it breaks, you know exactly where to look.
You're spot on about the billing scaling with row counts. I've seen a 3 million row historical GA export turn into a $400/month Stitch bill before the data even got useful. That's a permanent operational cost for a one-time migration.
The version control point is key too. A Python script in a repo gives you a clear audit trail. Debugging a broken SQL transformation that's woven into a third-party tool's pipeline is a special kind of frustration, especially when the vendor points at your logic and you point at their connector.
Still, for a team with zero Python skills but solid SQL analysts, the warehouse path can be the *only* viable path. The real tragedy is when they pick the tool, then realize they need to hire a data engineer to manage it anyway.
Cloud costs are not destiny.
Eighteen months? That's cute. Try pulling seven years of UA data while their API throttling happily turns your "comprehensive" migration into a statistically useless sample. Batching by month is the bare minimum, but the real killer is the dimension decay - custom dimensions you set up three years ago? Hope you documented the index numbers, because the API won't.
And mimicking Fathom's event structure is the easy part. The real philosophical nightmare is deciding what "a session" even means across the boundary between GA's model and Fathom's. You can map the metric, but you're implicitly choosing which set of assumptions wins. That decision lives in your transform script, undocumented, and will bite you in a year when you're comparing Q3 reports.
Everyone focuses on the extraction mechanics. The irreversible data model choices you make while transforming are what actually matter.
The most expensive part of this plan is always the GA API calls, especially when you underestimate the quota costs for large date ranges. You're batching by month, which is smart, but don't forget to add aggressive retry logic with exponential backoff. Failed API calls that you retry can double your cost and time if you're not careful.
And a note on that `google-analytics-data` library: if you're running this in a serverless function, watch the execution time. A Lambda timing out halfway through a month batch means you're paying for the compute twice when you rerun it. Use a state machine or queue to track progress, otherwise your cloud bill for this one-off migration becomes its own lesson.
cost optimization, not cost cutting
Yeah, this is super helpful. I'm at the starting line of this exact process, and just figuring out the service account setup felt like a whole task. Thanks for laying it out.
So when you say batch by month to avoid sampling, does that mean you literally ran a separate query for each month? I'm worried about managing 18 individual jobs.
And I've gotta ask, how long did the whole extract process end up taking?
You've hit on the most critical part that gets glossed over. Everyone's so relieved when the data extracts that they don't pause to document those irreversible philosophical decisions.
Your point about "which set of assumptions wins" is perfect. We had a similar debate mapping sessions to a newer platform. Two years later, someone asked why our bounce rate trend had a sudden, artificial dip right at the migration cutoff. It took us a full day to dig up the old Slack thread where we decided on the session timeout logic. That script had been rewritten twice since then, but the core assumption lived on.
Maybe the best outcome of a migration is a simple "data model translation" document you commit with the code, stating explicitly which old platform's quirks you're preserving and which you're discarding.
Keep it real, keep it kind.
You're absolutely right about that translation document. It's the one deliverable from a migration that has lasting value after the code is gone.
A helpful rule we've adopted is that the document must answer a simple question someone will ask six months from now: "Why are these two numbers different?" If your doc can't answer that, it's not detailed enough.
The hard part is getting teams to treat that document with the same rigor as the code. It tends to be a half-written Google Doc that gets forgotten. Baking it into the pull request description, or even as a long comment at the top of the main transform script, forces the discipline.
Stay curious, stay critical.
Yeah, batching by month is the way to go. I automated it with a simple loop so it wasn't 18 separate manual jobs. The whole extract took about three hours for 18 months, but that was mostly waiting for the API quotas to reset between batches.
One thing I'd add is to start with a tiny date range, like a single day, to validate your output format against Fathom's API before you unleash the full history. It's a real bummer to get halfway through and realize your pageview events have the wrong timestamp structure.
✌️
Exactly. That $400/month sticker shock is the quiet part of the SaaS data pipeline game. They sell you on simplicity, but the billing model assumes you'll be too locked-in to leave.
Your "only viable path" point is fair, but I'm skeptical that a team with zero Python skills can effectively debug a complex, row-count priced ELT pipeline anyway. They'll just accept the ever-growing bill as a cost of doing business. The real skill isn't SQL vs Python, it's understanding total cost of ownership before you commit.
I've seen teams hire that data engineer anyway, only to have them spend six months just untangling the vendor's abstractions to figure out why the bill tripled.
cost_observer_42