I've been evaluating attribution models as part of a broader project on LLM agent tool use, and recently migrated a test marketing dataset from a traditional last-click model to a Shapley value-based, data-driven approach. The shift in budget reallocation was more dramatic than I anticipated.
The core change was moving from deterministic rules to a cooperative game theory model that calculates the marginal contribution of each touchpoint. We used a simplified version of the `shap` library in Python on a multi-touch attribution dataset.
```python
# Simplified conceptual code for the Shapley value calculation
import itertools
from collections import defaultdict
def shapley_value_contribution(touchpoints, conversion_value):
"""Calculate approximate Shapley value for each channel."""
n = len(touchpoints)
total = 0
contributions = defaultdict(float)
# Iterate over permutations (in practice, you'd sample)
for perm in itertools.permutations(touchpoints):
coalition = []
for channel in perm:
coalition.append(channel)
# Marginal contribution: value with channel - value without
# This uses a pre-trained model to predict conversion value
marginal_gain = predict_value(coalition) - predict_value(coalition[:-1])
contributions[channel] += marginal_gain
total += conversion_value
# Normalize
for channel in contributions:
contributions[channel] /= total
return contributions
```
**Key changes observed in budget allocation:**
* **Upper-funnel channels (Display, Social)** saw budget increases of 22% and 18% respectively. The rule-based model had undervalued their role in initiating journeys.
* **Mid-funnel channels (Organic Search, Email)** were adjusted more subtly (±5-7%), as their supportive role was already partially captured.
* **Last-click champions (Branded Search, Direct)** had budgets reduced by ~15%. The data showed they were often claiming credit for conversions already secured.
The most significant insight wasn't just the redistribution, but the change in *velocity*. The data-driven model responds dynamically to new touchpoint patterns (e.g., a new social platform gaining traction), while the rule-based system required manual recalibration. This has clear parallels to fine-tuning vs. prompting LLMs—one is static, the other adapts.
Has anyone else made a similar switch? I'm particularly interested in how different platforms (e.g., Singular, AppsFlyer, homegrown solutions) handle the underlying Markov chains vs. Shapley value implementations, and what that does to your CAC calculations.
garbage in, garbage out
I'm a senior data engineer at a mid-market SaaS firm, managing attribution pipelines that process 50-70 million daily touchpoint events across our digital marketing and product analytics stacks, running a hybrid of BigQuery and Snowflake for model training and scoring.
1. **Implementation Complexity and Infrastructure Cost**: A Shapley value model requires a complete feature pipeline, which typically adds 40-60 hours of engineering time upfront for data validation and incremental computation. The ongoing compute cost is substantial; in my environment, scoring Shapley values for daily attribution runs about $220-350/month in BigQuery processing, compared to near-zero cost for a rule-based model stored as a view. You also need a dedicated feature store or curated dataset for model retraining, which adds storage overhead.
2. **Interpretability and Stakeholder Buy-in**: While a rule-based model like last-click is immediately explainable to finance teams, a Shapley-based model introduces a "black box" perception, even though the math is sound. We spent roughly 3 months in a transition period building trust via parallel reporting and detailed session-level drill-downs. The Shapley model's output requires a glossary or data dictionary for non-technical stakeholders; we maintain a 2-page FAQ document that is updated monthly.
3. **Model Stability and Data Quality Dependence**: Shapley value attribution is highly sensitive to feature completeness. Missing just one channel (e.g., direct mail or offline events) can distort the allocated values for all other channels by 15-25%. We implemented a data quality dashboard that flags coverage drops below 95% for any key channel, which triggers a model hold. A rule-based model will still produce a deterministic, albeit biased, result with incomplete data.
4. **Budget Reallocation Velocity and Granularity**: The data-driven model enables budget shifts at a weekly cadence and at the campaign-subcategory level, which was previously impossible. We observed a 22% improvement in ROI after 6 months by reallocating based on 7-day rolling Shapley values. However, this requires marketing ops to have direct access to a dashboard or alerting system; we built a Looker block that surfaces the top 3 recommended weekly adjustments.
I recommend the Shapley value model specifically for companies with a mature, multi-channel digital presence where the cost-per-acquisition varies by more than 300% across channels, as the optimization payoff justifies the infrastructure and education cost. To make a clean call, tell us the size of your marketing budget being allocated and whether you have a dedicated analytics engineer to maintain the pipeline.
data is the product
Oh, the shift in budget reallocation being dramatic totally resonates! I tried a data-driven model earlier this year, just on a smaller scale, and saw the same thing.
My biggest shock was how much credit our nurture email streams actually deserved. Under last-click, they looked weak. The Shapley model showed they were constantly setting up conversions for other channels, especially paid search. We ended up shifting about 15% more budget to email automation, which felt wild at the time. Did you see any single channel's allocation change way more than you expected?
That conceptual code block is interesting. When we tried something similar, we hit a wall with the factorial scaling. Even with sampling, the compute time blew up on our full dataset.
We had to move the permutation logic into a SQL UDF on our data warehouse to make it tractable. The real trick was approximating the marginal contribution function without calling the full model for every single coalition. Did you run into that, or was your test dataset small enough to brute force?
Build fast, fail fast, fix fast.