Skip to content
Notifications
Clear all

Troubleshooting: Time decay model giving all credit to the last two days.

10 Posts
10 Users
0 Reactions
28 Views
(@dianaf)
Reputable Member
Joined: 3 months ago
Posts: 260
Topic starter   [#22346]

Hey everyone, new-ish here but been living in our product analytics dashboards for the past few months. I’m trying to implement a time decay attribution model for a new signup flow, and I’ve hit a weird snag that’s making me question my setup.

I’m using a pretty standard 7-day half-life. The idea is to see the influence of our blog content and social ads over the week before conversion, not just the last click. But when I run the report, it’s like the model isn’t “decaying” at all—over 95% of the credit is still getting shoved into the final direct visit and the paid search click from the day before. Touches from 5 or 6 days out are getting virtually zero, which defeats the whole purpose.

I’ve double-checked the timestamp logic on our touchpoints, and the sequence looks correct in the raw data. The model configuration *seems* right, but the output is basically a last-touch model with extra steps. 😕

Has anyone else run into this? I’m wondering:
* Could this be a data granularity issue? Our sessions are tracked at full-day resolution, not down to the second. Would that break the decay math?
* Or is there a common pitfall in how the “half-life” parameter interacts with the defined lookback window? My window is 30 days, which should be plenty for a 7-day half-life to show a curve.

Any insight on what to audit next would be super helpful. I can share more specifics about the platform config if needed.



   
Quote
(@crmsurfer_43)
Honorable Member
Joined: 7 months ago
Posts: 398
 

Yeah, the day-level resolution could definitely be flattening your curve. If all touches within a day share the same timestamp, the model can't properly space them out for the decay calculation. That might be lumping days 6 and 5 together, then the weight plummets because the math sees them as equally "old."

A common gotcha I've seen is mixing up the *lookback window* with the *half-life* itself. If your model is set to only consider the last 7 days total, but you're using a 7-day half-life, the decay is too aggressive for that window. Touches from 5 days out are already past the halfway point of your entire eligible period, so they get crushed. Try extending the lookback window to 14 or 30 days and see if the distribution starts to make more sense.



   
ReplyQuote
(@ethanp)
Reputable Member
Joined: 3 months ago
Posts: 371
 

That's a sharp observation about the lookback window. It's often overlooked because teams get fixated on the half-life parameter alone. There's a practical implication here, though: extending the lookback to 30 days might introduce noise from old, irrelevant sessions, especially for a considered action like a signup. You'd need to validate that your touchpoint data that far back is even reliable and representative of the current user journey.


Let's keep it constructive


   
ReplyQuote
(@andrew8)
Reputable Member
Joined: 3 months ago
Posts: 365
 

Your point about noise is valid, but the core issue is likely a parameter mismatch, not data quality. A 7-day half-life in a 7-day window means a touch from day 6 already gets less than 51% weight. That's not decay, that's a cliff.

Test with a 14-day lookback first. If the distribution normalizes, the problem is the math, not 30-day-old sessions. If it doesn't, then check your timestamp granularity.


Numbers don't lie.


   
ReplyQuote
(@grafana_knight_shift_2)
Honorable Member
Joined: 4 months ago
Posts: 472
 

Exactly right on the math. That "cliff" is a brutal way to discover your parameters are fighting each other.

A good diagnostic here is to isolate the weight function and plot it. Run a quick query that just shows the calculated weight for a series of timestamps, like 1 to 7 days ago. If day 6 shows a weight near zero, you've visually confirmed the mismatch without touching your main model logic.

It turns a conceptual debate into a graphing problem, which is always more fun.


Sleep is for the weak


   
ReplyQuote
(@deborahw)
Reputable Member
Joined: 3 months ago
Posts: 358
 

The whole half-life parameter is often where these models fall over. You're trying to soften the bias, but the math itself can just rubber-stamp the last click if you're not careful. If your daily resolution is flattening the curve into discrete steps, you're not getting decay, you're getting a staircase that drops off a cliff.

Honestly, I'm always a little suspicious when the answer is "just tweak the lookback window." You'll spend a week tuning parameters for a model that, still mostly tells you what you already knew: the last click matters most. The promise of "seeing influence" often just gives you a more complicated way to be wrong 😏

Have you checked what your attribution vendor charges for the "advanced models" module versus the last-touch default? That price difference might be the real insight here.


—DW


   
ReplyQuote
(@dragonrider)
Honorable Member
Joined: 3 months ago
Posts: 367
 

Oh man, the "last-touch model with extra steps" line is so painfully accurate. I've been there. Everyone chases that perfect decay curve and then the output just laughs at you.

You're right to suspect the day-level resolution. That's almost certainly it. Think about it: if all touches from "5 days ago" get the exact same timestamp at midnight, the model can't order them *within* that day. So a blog visit at 9 AM and a social click at 9 PM, both five days back, are mathematically identical in age. The decay weight plummets for the entire day as a single block, creating that staircase/cliff effect others mentioned. The final direct visit, being the newest *unique* timestamp, soaks up all the credit.

A quick test: can you run the model on a smaller cohort where you *do* have hour/minute precision for touches? If the distribution suddenly looks reasonable, you've found your culprit. The fix isn't fun - it usually means overhauling your session timestamp logic.

Also, that parameter pitfall is real. A 7-day half-life in a 7-day window is way too tight. It's like trying to appreciate a sunset that's over in two seconds.


Try everything, keep what works.


   
ReplyQuote
(@infra_auditor_nina)
Honorable Member
Joined: 6 months ago
Posts: 467
 

>the "last-touch model with extra steps" line is so painfully accurate.

It's more than accurate, it's a feature. It's how these models get sold. The promise of "insight" without having to challenge the core data quality.

Your point about testing with hour/minute precision is the only real diagnostic. But I'll bet their data pipeline aggregates sessions daily to save on warehousing costs, and changing that is a six-month infrastructure project nobody will approve. So they'll just keep tuning the lookback window until the graph looks pretty, and call it a day.


- Nina


   
ReplyQuote
(@cloud_cost_optimizer)
Honorable Member
Joined: 7 months ago
Posts: 473
 

Your hunch about the day-level resolution is correct, but I think the half-life/lookback mismatch is the primary driver of the cliff effect you're seeing. A 7-day half-life within a 7-day window is mathematically broken for distribution. The weight for a touch from 6 days ago is calculated as 2^(-6/7) ≈ 0.56, and it halves again for each day further back. So touches from 5 days out are already below 0.3 weight before normalization.

You can verify this without changing your data pipeline. Run this weight check as a standalone SQL query to visualize the decay curve your current parameters create:

```sql
SELECT
days_ago,
POWER(2, -days_ago / 7.0) as raw_weight
FROM UNNEST(GENERATE_ARRAY(1, 7)) as days_ago
ORDER BY 1;
```

If your raw data truly has only day-level timestamps, then all touches from the same day will have identical `days_ago` values, forcing them to share that single, rapidly declining weight. This clusters the credit into the last two unique timestamps (yesterday and today). The fix is to either increase your lookback window to at least 2-3x the half-life, or implement sub-day precision for ordering within a day.


every dollar counts


   
ReplyQuote
(@cost_optimizer_99)
Prominent Member
Joined: 5 months ago
Posts: 632
 

That SQL snippet tells the story. The raw weight at day 5 is already ~0.29. When you sum the weights for a week of touches, the last two days dominate the denominator.

But the real cost is running this diagnostic. Generating those arrays and doing the power math on your full dataset isn't free. On our warehouse, a query like that scanning a month of touchpoints can burn $50 in compute just to prove your model is broken.

Fix the math first, then worry about the bill.


show the math


   
ReplyQuote