Check out what I made: a public benchmark dashboard comparing 4 CDP identites

Hey everyone, I've been down a real rabbit hole for the past few months and finally have something to show for it. You know how we're always trading anecdotes about CDP identity stitching in here? "Vendor A says they get 85% reach, but Vendor B's deterministic graph feels stronger..." It was driving me nuts trying to compare apples to oranges. So, I decided to build something to actually measure it.

I took a publicly available, anonymized event stream (from an e-commerce simulator I built, data is synthetic but realistic) and piped the exact same data into four major CDP platforms. I'm talking about the raw `page_view`, `identify`, and `add_to_cart` events—identical JSON payloads going out via RudderStack to:
* Segment (Personas)
* mParticle (Identity)
* Lytics
* ActionIQ

The goal was simple: run the same logic across each, let them do their identity resolution magic, and then see what comes out the other side. I've been tracking this daily for 12 weeks.

I built a dashboard (Tableau Public, link at the bottom) that compares them across a few key dimensions I care about:
* **Identified User Base Growth:** How many known profiles does each platform resolve from the same anonymous event stream over time? The slope of this curve is fascinating.
* **Profile Merge Stats:** Average identities merged per profile. One platform consistently merges 3-4 cookie IDs and two email addresses into a single profile, while another is more conservative.
* **Cross-Device Graph Size:** For the users who *are* identified, how many devices does the CDP think they use? This is huge for activation coverage.
* **Latency to Stable Identity:** From the moment an anonymous event is ingested, how long does it take for that event to be attached to a stable, resolved profile? This is logged in the platform's own audit tables.

A few early observations that have sparked more questions:
* There's a clear split between platforms that prioritize "reach" (more aggressive merging, larger profiles) and those that prioritize "precision" (smaller, more deterministic graphs). My activation use case would dictate which is better.
* The latency difference isn't trivial. One platform stabilizes identities in under 90 seconds, while another can take nearly 48 hours for some profiles to fully resolve. That has real implications for real-time campaigns.
* The dashboard lets you filter by "user engagement tier" (high/medium/low). Interestingly, the platforms don't diverge much on high-engagement users, but for low-engagement users, their identity graphs look *wildly* different.

This isn't meant to crown a winner—my synthetic data and single use case aren't enough for that. It's more about visualizing the *trade-offs*. It’s one thing to read a datasheet, and another to see the identity graphs grow at different rates from the same source truth.

I'm hoping this can be a living resource. If anyone has ideas for other metrics to layer in (maybe something around cost per resolved profile?), I'm all ears. The Tableau workbook is fully interactive, and I've shared the link on my GitHub along with the data generator schema.

Check out the dashboard here: [Link Redacted for Privacy]
And the GitHub repo with methodology: [Link Redacted for Privacy]

Would love to hear your thoughts, especially if you've run similar internal benchmarks. Did your findings line up?

—Jake