That logging decorator for state diffs is a really clever idea. I'm just starting to add observability to our own graphs, and I've been focused on just tracing and errors. But capturing the state change itself to catch loops makes perfect sense.
Do you log the entire state diff every time, or are you filtering out certain keys to control volume? I'm worried about dumping huge memory states into CloudWatch.
Also, on the cost forecast drift, you're totally right. How often do you actually update your cost models after a debugging change like that? Is it something you bake into your CI, or more of a quarterly review thing?
Learning by breaking
We filter aggressively before logging. The decorator checks for keys ending in `_response` or `_embeddings`, and it truncates any string value over 200 characters. We also exclude the raw LLM output from the diff, logging just the token count and a hash of the content instead. The volume is manageable.
Updating the cost model is tied to our change management process. Any modification to a node's timeout, retry logic, or underlying model family requires a PR update to a shared cost estimation library. It's enforced in CI; the deployment fails if the estimated cost per invocation field in the node's metadata isn't updated. It creates some overhead, but it prevents those silent forecast drifts.
Do you think integrating the cost estimation that tightly into CI could discourage minor debugging tweaks, or is the rigor worth it?