Skip to content
Notifications
Clear all

TIL: How to use a data lake as your identity resolution source of truth

2 Posts
2 Users
0 Reactions
3 Views
(@andrewh)
Estimable Member
Joined: 1 week ago
Posts: 85
Topic starter   [#12943]

Hey everyone, I've been trying to wrap my head around how a Customer Data Platform actually stitches customer data together. I always thought you *had* to use the CDP's built-in identity graph for everything.

But in my new role, our team showed me a different approach. We use our cloud data lake (Snowflake) as the single source of truth for customer IDs and their links. So, instead of the CDP trying to resolve identities on its own, we feed it a pre-built, unified customer profile table from the lake.

The main workflow is:
1. All raw event and customer data lands in the lake.
2. We run our own SQL-based identity resolution logic there, merging records from different sources based on our business rules.
3. We export this single "master customer profile" table to our CDP.
4. The CDP then uses those pre-resolved IDs for segmentation and activation.

This was a lightbulb moment for me. It means we have one consistent customer view for both analytics in the lake *and* for marketing campaigns in the CDP. Has anyone else set things up this way? I'm curious about the pros and cons you've seen, especially for a beginner like me trying to understand data flow control.



   
Quote
(@charlotte0)
Estimable Member
Joined: 1 week ago
Posts: 72
 

That's a really interesting approach to centralizing identity logic. In my work with HR systems, we face a similar problem trying to unify employee records from multiple payroll and benefits providers.

A potential drawback I'd consider is latency. Since your workflow involves a process in the lake before syncing to the CDP, how do you handle scenarios requiring real-time identity resolution, like a customer service interaction? The consistency benefit is clear, but you might need a separate, faster path for those use cases.

Has your team found the SQL-based logic easier to audit and modify over a vendor's black-box graph? That control sounds appealing.



   
ReplyQuote