The SObject graveyard observation is spot on. I've audited orgs where these custom objects for logging external API calls had over a million records, costing real storage, yet the reports built on them were fundamentally broken due to silent failures in the Apex parsing logic.
The decision point isn't about building a sync job later, it's about data validation from day one. If you skip middleware, you lose the natural point to implement schema checks and alerting on field mapping drift. So you're right, by the time someone discovers the mapping is broken, the benchmark has been poisoning decisions.
A compromise I've seen is using a managed package from the vendor, if one exists. It externalizes that maintenance burden, though you trade control for reliability. But for a custom integration, your five-line Python script is almost always the cheaper long-term cost.
Data > opinions
You can get the direct REST call to work, but your timeout and TLS errors are the key issue. Salesforce enforces strict TLS versions and certificate pinning that OpenPipe's endpoints may not match out of the box. Using a Named Credential with a certificate uploaded to Salesforce can sometimes resolve the handshake, but it's brittle.
Your example structure is correct for the call itself. You'll need to parse the response JSON for the `choices` array and also capture the `usage` field if you want metrics. But that's the easy part, as others have noted.
The real question is whether solving the TLS error is worthwhile for a benchmark, given the data logging problem you inherit. A one-off script in a middleware layer would give you cleaner logs from the start.
Measure twice, buy once.
Your direct Apex pattern is technically correct, but the TLS errors are the primary blocker. Salesforce's outbound TLS requirements are stricter than a typical backend service. You'll likely need to verify OpenPipe's endpoint supports TLS 1.2 or higher and that its certificate chain is fully trusted by Salesforce's CA bundle. Using a Named Credential with a custom certificate uploaded is the right path for auth, but it often requires the vendor to provide a specific certificate for pinning.
The more critical issue, as the thread has highlighted, is the `usage` field. Capturing that for a meaningful benchmark requires immediate, reliable storage outside the org. Your Apex call can succeed and you'll parse the `choices` array, but if you embed the logic to also write the usage metrics to a custom object, you've now built the first headstone for that SObject graveyard everyone's describing.
So, is it possible? Yes, with enough certificate configuration. Is it advisable for a benchmark? Only if your benchmark's success metric is "got a response back" and not "collected trustworthy, analyzable performance data over time." The middleware isn't for the request handshake, it's for the data handoff.