Your methodology is sound. The key metric is the drop from 95% to 42% initiation. That's a classic decay curve signaling either immediate value wasn't perceived or the activation energy was too high.
Quantifying the context switch cost at 3.5 hours is crucial. Most internal ROI calculations only account for the purported time *saved* by the tool, not the time *lost* to its integration friction. That delta often kills the business case when you model it at scale.
I'd be curious about your instrumentation. Did you capture events directly from the IDE plugin's own APIs, or was this based on manual survey data? The former gives you a cleaner latency measurement for the "time to first value" after opening the tool, which often explains the week-two drop-off.
numbers don't lie
I completely agree about the 3.5 hour context switch cost being the real poison pill. We model that as a direct sunk cost against the tool's annual license fee. If a tool requires more than eight hours of setup and context switching across the team, the first-year ROI is almost always negative.
Regarding instrumentation, we used a hybrid approach. The IDE plugin events gave us the precise "time to first value" latency, which averaged 22 minutes just for the plugin to initialize its own cache. But we paired that with a one-question survey triggered on the third use: "What were you trying to do before opening this tool?" That's how we captured the *cognitive* switch cost, which the API logs can't show. The 3.5 hours came from aggregating the interrupted tasks people reported - it wasn't just the tool's latency, but the time to refocus on their original work.
That correlation between high initial latency and the week-two drop-off was almost perfect. If the first interaction took longer than five minutes, subsequent usage plummeted. The vendors never track that.
RTFM — then ask for the audit