I've been poking at this integration for a few weeks, trying to get Claude's API into an automated workflow. The official Zapier Claude integration is basically just their web interface in a box—it doesn't expose the API directly. That means you're stuck with their latency and you can't send custom spans or metrics.
If you want *real* integration with observability, you have to go the API route and build your own Zapier step via webhooks. Here's the basic flow I've tested:
* Use a Zapier Webhook trigger to catch an event.
* Send that payload to a custom middleware (I use a tiny Go service) that instruments the call.
* The middleware calls the Claude API, adds tracing headers, and logs the timing.
* Returns the result back to Zapier for the next action.
The latency is entirely dependent on that middleware and Claude's API response time. In my tests, for a simple completion, the total round-trip from Zapier trigger to action was consistently between 1.8 and 2.5 seconds. The Claude API portion alone was 1.2 to 1.7 seconds.
The critical thing missing here is native OpenTelemetry support. You can't see the trace from the initial Zapier trigger through Claude's internal processing. Without that, you're blind to where the real delays are happening. You're just measuring black-box latency.
If you're going to build this, instrument everything. Here's a snippet from my middleware to show the timing capture:
```go
ctx, span := otel.Tracer("claude-zapier").Start(ctx, "claude-api-call")
defer span.End()
start := time.Now()
resp, err := claudeClient.CreateMessage(ctx, request)
duration := time.Since(start)
span.SetAttributes(attribute.Int("completion.tokens", resp.Usage.OutputTokens))
span.SetAttributes(attribute.Float64("duration.ms", duration.Seconds()*1000))
```
Without this, you're just guessing. Has anyone else built a more direct integration and managed to get actual distributed traces working?
-ninja
Observability is not monitoring
Your latency numbers align with my own API tests using Claude 3 Opus. That 1.2-1.7 second range for the API call is typical for a non-trivial prompt, but it can spike unpredictably.
The middleware approach you describe is the only way to get proper observability right now. I built a similar proxy in Python to capture token usage and latency percentiles. The variance is the real issue - in a batch of 100 requests, I saw p95 latency climb to over 4 seconds, which can break time-sensitive Zaps.
Have you tried benchmarking against the official Zapier-Claude step? I'm curious if their packaged integration has better consistency, even if you lose the custom metrics.
Exactly. That 1.2 to 1.7 second range for the Claude API call is just the happy path. The total latency you're measuring is mostly network overhead and serialization, which is fine until it isn't.
Your missing OpenTelemetry support is the real killer. Without native tracing from Anthropic's side, you're only instrumenting your own proxy. You have no visibility into Claude's internal queueing, model loading, or token generation spans. You see a black box taking 1.7 seconds, but you can't tell if it was network latency to their endpoint, cold starts, or genuine compute time.
I'd be curious to see your p99 latency over a week. My bet is those 2.5-second completions will occasionally stretch to 8 seconds, and your middleware logs will blame Claude while offering zero diagnostic evidence.
P99 or bust.
Your middleware approach is the correct pattern, but you're right that the visibility stops at the API boundary. I'd push you to add more dimensions to your logging beyond just timing. Capture the model version, the exact input token count, and the output token count for each request. That high-cardinality data, when charted against latency, often reveals patterns the simple p95 misses, like latency cliffs at specific token thresholds.
The real issue, as you've identified, is the lack of upstream traces. You can mitigate this slightly by treating Claude's API as an external service in your tracing. Instrument your outgoing HTTP call with attributes for the full prompt hash and the Anthropic request ID from their response headers. When a user reports a slow completion, you can at least correlate their experience with your proxy's timing and the specific request ID for Anthropic support, moving from "it was slow" to "request ID `req_123` had 2.1s proxy latency for a 512-token prompt."
Have you considered adding a synthetic metric for timeouts? Since you control the proxy, you could define a service-level objective for your Claude calls, say 2.5 seconds, and increment a counter for violations. That would give you a clearer business metric than just observing latency percentiles in a dashboard.
you can't fix what you don't measure
You're right about the lack of upstream traces being the core observability gap. I'd add that the Anthropic request ID you mentioned is crucial, but it's only useful for their support team during a ticket. It doesn't help you perform your own root cause analysis across hundreds of requests in your dashboard.
One mitigation we've implemented is to treat latency spikes as a function of input/output token ratio. We log that ratio for every call and set different SLOs for different bands. A prompt under 100 tokens with a 2000-token response gets a much more lenient latency threshold than a simple Q&A. It's not true tracing, but it creates a performance model that makes the black-box behavior more predictable for alerting.