We just moved off a "proper" CDP vendor after our contract renewal came in 3x higher. The demo-to-production performance gap was massive, especially on identity resolution at our scale (~2M monthly web users, 500k mobile app users). The vendor's black-box processing was a liability for our analytics team.
I built a replacement over three weeks using RudderStack (Cloud) and Snowflake. Total cost is under $5k/month, and we own the entire data model. The core setup:
* **RudderStack Cloud** for event collection (Sources: Web, iOS, Android, Server). We use their JavaScript and React Native SDKs. Configuration is done via their UI, but here's the crucial part of our `rudderlab.js` for web:
```javascript
rudderanalytics.load("YOUR_WRITE_KEY", "YOUR_DATA_PLANE_URL");
rudderanalytics.identify(
"user_123",
{ email: "[email protected]", tier: "premium" },
{
integrations: { All: false, Snowflake: true },
context: { campaign: { name: "spring_2024" } }
}
);
```
We route everything directly to Snowflake, bypassing any secondary cloud storage to keep latency down.
* **Snowflake** as the processing layer. We use a MEDIUM warehouse for transformations. The key is using Snowflake tasks and streams to handle the identity graph. A simplified version of our stitching logic:
```sql
CREATE OR REPLACE TASK cdp_user_stitch
WAREHOUSE = 'cdp_wh'
SCHEDULE = '5 minute'
AS
MERGE INTO unified_customer_profile t
USING (
SELECT
COALESCE(identified_user_id, anonymous_id) as master_id,
MAX(email) as email,
-- ... other rules
FROM rudderstack_events
WHERE _timestamp >= DATEADD(hour, -24, CURRENT_TIMESTAMP())
GROUP BY 1
) s ON t.master_id = s.master_id
WHEN MATCHED THEN UPDATE SET t.email = s.email
WHEN NOT MATCHED THEN INSERT ...;
```
* **Hightouch** for reverse ETL to sync segments back to Salesforce, HubSpot, and ad platforms. This is where the "activation" piece lives.
**Who this works for:** B2C SaaS or E-commerce with an existing data team (even 1-2 engineers). You need someone who can write SQL and maintain a few pipelines. If you're under 1M events/month, RudderStack's free tier plus Snowflake credits might keep you under $1k.
**Performance:** Our main identity resolution job runs every 5 minutes. Latency from event to actionable profile is under 7 minutes, which beats the 30+ minute delays we were seeing. Cost is predictable and scales with Snowflake usage.
The trade-off is clear: you get control and cost predictability, but you're on the hook for building and maintaining the logic. For us, that was preferable to paying a premium for a system we couldn't debug.
Show me the query.
Your point about the performance gap between demo and production resonates. Many proprietary CDPs optimize their staging environments with clean, synthetic data that doesn't reflect real-world complexity, particularly around identity graph maintenance.
While your approach solves for cost and transparency, have you validated the recall and precision of your Snowflake-based identity resolution compared to the vendor? At your scale, even a 2% degradation in accuracy can materially impact downstream campaign performance. A controlled A/B test on a segment, using the old system's output as a baseline, would be a prudent check.
prove it with data
That's a huge win on cost and transparency! The black-box identity resolution from vendors always made me nervous too.
We did something similar but used dbt for the transformations inside Snowflake. Our identity stitching logic lives as a series of dbt models, which makes it version-controlled and way easier for the analytics team to tweak. Something like this in a `user_mapping` model:
```sql
with user_events as (
select
user_id,
email,
anonymous_id,
min(timestamp) as first_seen
from {{ ref('rudderstack_events') }}
where user_id is not null or email is not null
group by 1,2,3
)
-- stitching logic here
```
Have you noticed any latency issues with routing directly to Snowflake, or is it pretty snappy? Also, what's your plan for managing schema drift from new events?
Infrastructure as code is the only way