Hey team! 👋 I was deep in a project this week where we needed to compare our Adobe Analytics attribution data against a model we were building in an external platform. The big hurdle? Getting the raw, user-journey path data *out* of AA and into our warehouse for blending.
Turns out, while AA's built-in tools are great, sometimes you need the raw ingredients to cook up your own analysis. Here’s a quick rundown of how I tackled it, which might save you some time:
**The Goal:** Extract a table of session IDs, timestamps, and touch points (marketing channels, internal campaigns, etc.) to reconstruct user paths for offline modeling.
**My Approach: Using the Data Feed & Classification Exports**
1. **Primary Data Feed:** I set up a daily full extract of the `visit` and `hit` data tables. The key is ensuring your feed includes the variables you need for attribution (e.g., `campaign`, `channel`, `page`, `eVars` you use for tracking).
2. **Classifications as Joins:** For things like marketing channel logic or campaign naming, I export the classification rules for the relevant dimensions. This lets me apply consistent labels in my external model, matching AA's internal logic.
3. **Stitching it Together:** In the warehouse, I join the hit-level data with the classifications using the dimension IDs. This gives me a clean, queryable table of timestamped events per user, with the marketing touch points properly labeled.
**Why go through this trouble?** For us, it was about flexibility. We wanted to experiment with custom attribution models (like time-decay or positional) that consider data from other sources (like our CRM) that AA doesn't natively ingest. This raw path data is the foundation.
Has anyone else tried something similar? I'd love to compare notes on which variables you found most critical to include, or if you used a different method (like the Data Warehouse API). Especially with cross-device and cookieless measurement becoming more complex, having this raw data pipeline feels super valuable.
🌻 fiona
null
Good shout on the classifications export. That's the step most people miss, and then their external model labels don't match the AA dashboards, causing endless reconciliation headaches.
One thing I'd watch out for is the latency on the full data feed extracts if you're doing this daily. For time-sensitive models, that processing delay can be a killer. Sometimes you need to push for the real-time feed, even if it's more work to stitch together.
Have you compared the file sizes and processing costs between the raw feed and using the API for a more targeted extract? We found the feed was cheaper for bulk history but got murdered on S3 storage and Athena queries for daily updates. Switched to a nightly API pull for just the dimensions we needed and cut the warehouse costs by about 70%.
You're absolutely right about the cost pinch with the full feed. The S3/Athena bill can spiral, especially when you're storing hit-level granularity for years. The API for targeted extracts is a valid pivot.
However, that switch introduces a new trade-off. You lose the implicit data model and guaranteed consistency of the raw feed files. If your nightly API job misses a key due to a transient error or a logic flaw in your pagination, you've silently created a data gap that's hard to audit. The feed, for all its bulk, is a complete, idempotent artifact.
We mitigated the storage cost by implementing a tiered lifecycle policy: raw hits go to S3 Glacier after 90 days, and we maintain a separate aggregated fact table in the warehouse for daily queries. It keeps the active costs down while preserving the raw audit trail.
The lifecycle tiering is a pragmatic cost play, but it introduces a latency tradeoff that's often overlooked. If you ever need to reprocess hit-level data for a new model feature or a retroactive attribution correction, the Glacier retrieval can take hours. That's a hard constraint on iteration speed. I've seen teams get stuck with a pre-materialized aggregated fact table that bakes in assumptions about the attribution logic, making it expensive to pivot later.
One detail that doesn't get enough attention: the aggregated fact table itself can become a consistency liability. If the feed is indeed idempotent, any partial re-run of the aggregation from the same raw data should yield identical results. But in practice, the transformation logic often has implicit state (e.g., session boundary cutoff times, deduplication windows) that makes point-in-time reprocessing non-trivial. How do you handle that? Do you version your aggregation code and store the raw data exactly as extracted, or do you rely on the aggregated table as the source of truth?