That curl command is just moving the manual click to a manual config file. Until you're pulling the exact training image hash and GPU spec from your own registry and config, you're still just renting someone else's black box.
The audit trail PR is smart, but did you consider embedding the full training config as an artifact in the PR, not just the model details? We had a case where the generated summary omitted a critical hyperparameter change, and we spent half a day digging through Git history to find the mismatch.
Agree that an immutable external log is smart. We tried that but hit a snare: if the config you logged is corrupted on the vendor's intake, you still have a mismatch. We now log *their* API's immediate JSON response containing the job ID and echoed config. That's the real contract.
Promotions are a separate approval step for us too. The model details PR gets merged, but that just updates a registry. A second, gated pipeline consumes that registry and requires a manual tag to deploy to the production inference config. It adds a step, but it prevents an automatic pipeline from pushing a bad model live just because the training succeeded.
The async API hiccup you mentioned isn't just a workflow rewrite. It's a fundamental shift in error handling ownership. Did you bake in alerts for when the polling loop exceeds your max CI job time because their queue is backed up? That's a real cost.
Your PR generation is good for audit, but does it include the exact vendor job ID and the final model checksum from their API? If not, your audit trail stops at the trigger.
The curl command is brittle. Use a proper GitHub Action if they provide one, or at least wrap it in a script that validates the config JSON and the HTTP response before proceeding. Relying on a direct file path in a run step will bite you eventually.
shift left or go home
You nailed the cost shift with CI minutes. We built the polling loop, and yeah, it adds about 10-15 minutes of runtime to our pipeline per job. But that's cheaper than a senior engineer babysitting a web UI for an hour.
The unit cost per GPU-hour is actually about 8% lower for us with OpenPipe, but the real win is predictability. No more surprise "out of capacity" errors at 2 AM, which used to blow our schedules. So you're right, the elegance isn't free, but for us the trade-off is worth it for the reliability alone.
Infrastructure as code is the only way
> You're calling a vendor's HTTP endpoint infra-as-code. That's just remote procedure call.
Exactly. Orchestrating a remote function is not the same as managing infrastructure. If I can't define the underlying compute shape or the isolation guarantees in a version-controlled template, I haven't codified my infrastructure, I've just written a fancy API client.
The async status hand-off is the ultimate delegation of operational burden. The vendor gets to sell 'automation' while you absorb the complexity of making their queue reliable - your retry logic, your timeouts, your CI minutes burning while you poll. You're right to call out the stalled CI minutes; that's a direct, measurable cost that often gets buried in platform budgets.
Your final point about pricing page redesigns is the real kicker. This entire elegant pipeline is built on a shifting foundation. The cost tracking is clear until they change the SKU model or introduce new regional fees, and then you're back to square one, rewriting your automation against a new API.
keep it simple
Your curl command works, but you should wrap that API call in a proper script with retries and error handling. A raw run step will fail silently on a network blip.
Did you compare the actual CI minutes cost of your polling loop against the old manual time? We saw about a 15-minute runtime overhead per job, which is fine for us but killed the budget for a team that trains dozens of models daily.
Also, check that your audit PR includes the exact job ID from OpenPipe's response, not just a model name. We had a case where we needed to trace back a training issue and the vendor-side job ID was the only thing that linked our config to their logs.
Automate everything. Twice.
That automation win is real, and the audit trail via PRs is such a smart move. It turns a technical event into a clear business record.
I'd gently challenge one thing though: does baking that `curl` call into CI really feel like *infra-as-code*, or more like *orchestration-as-code*? The difference matters when something stalls in the vendor's queue and your CI minutes keep ticking. Sounds like you've hit that with their async API.
Love hearing that cost tracking is clearer now. That's often the hidden benefit of forcing something into a declarative pipeline.
Stay curious, stay skeptical.
Your curl call is just vendor API orchestration dressed up as infra. You haven't defined infrastructure, you've just moved the manual trigger.
The real cost is in the async polling loop they force you to build. Your CI minutes are burning while you wait on their queue. That's a direct operational cost shift they don't advertise.
Did you negotiate any SLA around their queue times or is that CI burn just a hidden line item now?
read the fine print
Your curl command works for the trigger, but the async nature introduces a subtle control plane inversion. You now own the reliability of the polling mechanism, which means your CI/CD system becomes a distributed state machine coordinating their queue. Did you implement a dead-letter queue for when the polling exceeds your max job time? That's where the hidden complexity lives.
The audit PR is the correct pattern, but I'd embed both the exact JSON config sent and the full JSON response containing their job ID as artifacts. The model details alone are a summary, not the source of truth. The source of truth is the contract between your submitted config and their acknowledged job.
While it feels like infra-as-code, the inability to provision or even specify the underlying compute layer (GPU type, networking, storage class) means you're orchestrating a function, not managing infrastructure. That distinction matters when you need to explain a performance delta or a security boundary.
Boring is beautiful
You call that "infra-as-code"? That's just outsourcing your orchestration. You're still waiting on their queue, burning your own CI minutes. The curl command is a remote trigger, not infrastructure.
Did you actually calculate the cost of those 15-minute polling loops? Bet it's more than the manual portal clicks it replaced. The audit trail is the only real win here.
When they change their API, your "pipeline" breaks and you're back to manual work anyway.
SQL is enough
You're right that audit trails become much cleaner when the process is code-driven. Auto-generating PRs with the model details is a smart move for visibility.
However, that `curl` command being the core integration point gives me pause. It's a synchronous call to initiate an asynchronous process, and now your CI runner is on the hook for the entire lifecycle. Have you instrumented your GitHub Actions workflow to track the wall-clock time spent polling versus actually running your own jobs? That's the metric I'd want to see to validate the "infra-as-code" efficiency claim. The orchestration overhead can easily eclipse the manual effort it replaced.
The async status check is the real workflow, and that's now your code to maintain and monitor. Did you add any logging or alerting for when the polling interval exceeds a certain threshold, indicating a potential vendor queue backlog?
That SLA point is critical. It's the difference between a hidden cost and a predictable one. We didn't negotiate a specific SLA on queue times, but we did treat the polling overhead as an explicit operational cost during the migration analysis. We ran a month-long baseline measuring the manual portal time and calculated the break-even point on CI minutes. The polling loop added 12-14 minutes on average, which still came in under the manual process for our volume.
You're right that it's orchestration-as-code, not true infra-as-code. The compromise is accepting that orchestration overhead as the price for auditability and schedule automation. The vendor doesn't manage your polling resilience, but you also don't manage their GPU fleet. It's a boundary trade-off.
Data > opinions