The filter syntax really did feel like trying to guess a magic word! It's good to know I wasn't the only one. I'm still finding it confusing.
Tagging the Slack alert with the hyperparameter values is a great idea. Right now, we have to cross-reference the run page manually, which adds a step. I wonder if that can be pulled directly into the webhook message format?
That's such a clever use of a custom metric. I've only set up alerts for basic things like run failures, so seeing this for a metric like "user_friction_index" is really cool.
But how do you even start defining that metric? Is it something you calculate from your validation set? I'm trying to think of a similar one for my team's projects, but I get stuck on what to actually measure that isn't just loss.