Skip to content
Notifications
Clear all

Migrated from Hotjar to FullStory - 3 month deployment post-mortem for a mid-market e-commerce

2 Posts
2 Users
0 Reactions
4 Views
(@sre_geek)
Active Member
Joined: 1 month ago
Posts: 11
Topic starter   [#1711]

Our migration from Hotjar to FullStory was completed three months ago, and I believe we have now gathered sufficient operational data to perform a meaningful post-mortem, specifically on the pricing and licensing implications. The decision was driven by a need for more granular session replay and better integration with our existing event-driven architecture. However, the shift from Hotjar's relatively simple per-session model to FullStory's more nuanced consumption-based pricing revealed several unforeseen cost dynamics that I suspect are common in the analytics vendor space.

The core of the issue lies in the transition from a **session-centric** to a **data-usage-centric** model. Hotjar's pricing was based primarily on the number of recorded pageviews/sessions per month, which, while potentially expensive at scale, was predictable. FullStory, in contrast, employs a model based on **captured event volume**, measured in "events captured per month." This includes every DOM mutation, network call, click, and console log we choose to ingest. Our initial estimates, based on extrapolated session counts, proved to be significantly inaccurate.

We made two critical miscalculations:
* **Event Density:** We failed to account for the default event capture richness of FullStory. A single user session on our complex product pages generates an order of magnitude more events (e.g., every mouse movement, element visibility change) than the equivalent "session" in Hotjar.
* **Instrumentation Sprawl:** As developers, once we had access to the powerful API, we began instrumenting more custom business and performance events for deeper analysis. This directly and linearly increased our monthly event count.

A concrete example: our "Add to Cart" flow. In Hotjar, it was a recorded click. In FullStory, it became a composite of:
1. A custom `Business:CartAdd` event.
2. Dozens of associated DOM read/mutation events from the UI update.
3. Network events for the API call.
4. Console logs from the service worker.

This richness is valuable, but it directly translates to cost. We breached our contracted monthly minimum within the first 45 days, triggering overage fees.

Our current strategy to manage this involves aggressive data sampling and filtering at the source. We've moved from full capture to a targeted approach, using FullStory's configuration to exclude noisy, low-value events and to sample sessions based on user segment. Here is a simplified snippet of our current initialization configuration, which has stabilized our monthly event volume:

```javascript
window['_fs_debug'] = false;
window['_fs_host'] = 'fullstory.com';
window['_fs_script'] = 'edge.fullstory.com/s/fs.js';
window['_fs_org'] = 'XXX';
window['_fs_namespace'] = 'FS';
window['_fs_sampling'] = 0.5; // 50% session sampling rate
window['_fs_cookie_secure'] = true;

FS('set', {
'sitewide_elements': {
'exclude': [
'[data-testid="recommended-products"]', // Exclude high-churn, low-insight components
'[class*="advertisement"]',
'.chat-widget *'
]
},
'metric': {
'include': ['Business:*', 'Error:*', 'Performance:CLS', 'Performance:LCP'] // Only include specific custom metric classes
}
});
```

The hidden costs extended beyond raw event overages:
* **Increased storage footprint:** The richer sessions consume more data, impacting our internal log aggregation and archiving costs.
* **Compliance overhead:** With more granular data captured, our data retention and PII scrubbing processes became more complex and resource-intensive.
* **Performance tax:** While minimal, the client-side capture script has a non-zero performance impact, which required us to adjust our Core Web Vitals SLOs and monitoring.

Ultimately, the migration has provided superior diagnostic capabilities for our support and product teams, but the total cost of ownership is approximately 2.3x our initial projection. The lesson is that with powerful analytics platforms, the licensing model is intrinsically tied to implementation discipline. A "capture everything" approach is financially unsustainable; the system must be treated as a monitored service itself, with clear SLIs (e.g., event capture rate, cost per session) and SLOs. I am interested to hear from others in the mid-market space about how you've structured your contracts—do you negotiate hard caps, sliding scales, or employ other technical mitigations to keep consumption-based pricing predictable?


Error budgets are for spending.


   
Quote
(@new_reviewer_kyle)
Eminent Member
Joined: 3 months ago
Posts: 16
 

Hey, really appreciate you posting this. I'm Kyle, a growth marketer at a DTC brand doing about $15M in revenue. We run a similar stack (Segment, Shopify Plus, Klaviyo) and evaluated both Hotjar and FullStory last year for session replay and product analytics.

Here's my take on the key differences, based on that eval and running Hotjar in production:

**Real Cost:** Hotjar is predictable ($32/mo-$389/mo flat). FullStory's consumption model is elastic; their "Digital Experience Intelligence" pricing gets expensive fast as you add custom events. At our scale, a FullStory quote was 4x our Hotjar bill once we factored in our average sessions and event volume.
**Data Fidelity:** FullStory wins for technical debugging. The session replay detail, especially for JS errors and network activity, is in a different league. Hotjar's replays are simpler and sometimes glitchy, but they're good enough for basic UX review.
**Setup & Integration:** Hotjar is a snippet. FullStory requires more planning. You need to decide what events to capture upfront to avoid cost overruns; integrating it with our event stream via Segment was a solid two-day project.
**Use Case Fit:** Hotjar is perfect for SMB/mid-market teams who want quick, visual feedback on user behavior. FullStory is for product/engineering teams at mid-market+ who need to diagnose specific, complex user issues and have the budget for granular data.

If your primary goal is watching session recordings to spot UX friction, I'd stick with Hotjar. The cost certainty is huge. Only go FullStory if you *need* the technical replay depth for debugging and have clear guardrails on event capture. Could you share your average monthly sessions and if your team is more marketing or engineering-led? That would make the call easy.



   
ReplyQuote