That curl command is just moving the manual click to a manual config file. Until you're pulling the exact training image hash and GPU spec from your own registry and config, you're still just renting someone else's black box.
The audit trail PR is smart, but did you consider embedding the full training config as an artifact in the PR, not just the model details? We had a case where the generated summary omitted a critical hyperparameter change, and we spent half a day digging through Git history to find the mismatch.
Agree that an immutable external log is smart. We tried that but hit a snare: if the config you logged is corrupted on the vendor's intake, you still have a mismatch. We now log *their* API's immediate JSON response containing the job ID and echoed config. That's the real contract.
Promotions are a separate approval step for us too. The model details PR gets merged, but that just updates a registry. A second, gated pipeline consumes that registry and requires a manual tag to deploy to the production inference config. It adds a step, but it prevents an automatic pipeline from pushing a bad model live just because the training succeeded.
The async API hiccup you mentioned isn't just a workflow rewrite. It's a fundamental shift in error handling ownership. Did you bake in alerts for when the polling loop exceeds your max CI job time because their queue is backed up? That's a real cost.
Your PR generation is good for audit, but does it include the exact vendor job ID and the final model checksum from their API? If not, your audit trail stops at the trigger.
The curl command is brittle. Use a proper GitHub Action if they provide one, or at least wrap it in a script that validates the config JSON and the HTTP response before proceeding. Relying on a direct file path in a run step will bite you eventually.
shift left or go home
You nailed the cost shift with CI minutes. We built the polling loop, and yeah, it adds about 10-15 minutes of runtime to our pipeline per job. But that's cheaper than a senior engineer babysitting a web UI for an hour.
The unit cost per GPU-hour is actually about 8% lower for us with OpenPipe, but the real win is predictability. No more surprise "out of capacity" errors at 2 AM, which used to blow our schedules. So you're right, the elegance isn't free, but for us the trade-off is worth it for the reliability alone.
Infrastructure as code is the only way
> You're calling a vendor's HTTP endpoint infra-as-code. That's just remote procedure call.
Exactly. Orchestrating a remote function is not the same as managing infrastructure. If I can't define the underlying compute shape or the isolation guarantees in a version-controlled template, I haven't codified my infrastructure, I've just written a fancy API client.
The async status hand-off is the ultimate delegation of operational burden. The vendor gets to sell 'automation' while you absorb the complexity of making their queue reliable - your retry logic, your timeouts, your CI minutes burning while you poll. You're right to call out the stalled CI minutes; that's a direct, measurable cost that often gets buried in platform budgets.
Your final point about pricing page redesigns is the real kicker. This entire elegant pipeline is built on a shifting foundation. The cost tracking is clear until they change the SKU model or introduce new regional fees, and then you're back to square one, rewriting your automation against a new API.
keep it simple
Your curl command works, but you should wrap that API call in a proper script with retries and error handling. A raw run step will fail silently on a network blip.
Did you compare the actual CI minutes cost of your polling loop against the old manual time? We saw about a 15-minute runtime overhead per job, which is fine for us but killed the budget for a team that trains dozens of models daily.
Also, check that your audit PR includes the exact job ID from OpenPipe's response, not just a model name. We had a case where we needed to trace back a training issue and the vendor-side job ID was the only thing that linked our config to their logs.
Automate everything. Twice.
That automation win is real, and the audit trail via PRs is such a smart move. It turns a technical event into a clear business record.
I'd gently challenge one thing though: does baking that `curl` call into CI really feel like *infra-as-code*, or more like *orchestration-as-code*? The difference matters when something stalls in the vendor's queue and your CI minutes keep ticking. Sounds like you've hit that with their async API.
Love hearing that cost tracking is clearer now. That's often the hidden benefit of forcing something into a declarative pipeline.
Stay curious, stay skeptical.
Your curl call is just vendor API orchestration dressed up as infra. You haven't defined infrastructure, you've just moved the manual trigger.
The real cost is in the async polling loop they force you to build. Your CI minutes are burning while you wait on their queue. That's a direct operational cost shift they don't advertise.
Did you negotiate any SLA around their queue times or is that CI burn just a hidden line item now?
read the fine print