I've been testing Claw's API for generating pipeline configs from natural language. It's fundamentally broken for CI/CD use because the output is non-deterministic. Same system prompt, same user request, different YAML every time.
I'm using a simple, specific prompt:
```
System: You are a CI/CD expert. Output only valid GitHub Actions workflow YAML.
User: Create a workflow that runs unit tests on Python 3.9 and 3.10 on ubuntu-latest for pushes to main.
```
Here are three consecutive raw outputs (abbreviated):
**Run 1:** Used `actions/setup-python@v4`, matrix strategy for Python versions.
**Run 2:** Used `actions/setup-python@v2`, two separate jobs (no matrix).
**Run 3:** Added a pointless "Install dependencies" step with `pip install -r requirements.txt` even though I never mentioned a requirements file.
This isn't a minor formatting difference. It's a different architecture each time. If I tried to automate this in my toolchain, I'd get flaky, unreproducible infrastructure.
* How can you trust a generated pipeline if it changes every time you run it?
* This makes "AI-generated CI" a non-starter for anything serious. You can't review a moving target.
Has anyone found a pattern or a configuration parameter that actually locks this down? I've tried setting `temperature=0` in the API call, but it doesn't seem to respect it, or the variability is coming from somewhere else.
My current stack can't integrate a tool that introduces randomness into build definitions. I'm looking at this as a potential time-saver for junior devs, but if it's inconsistent, it's worse than useless—it's dangerous.
Build once, deploy everywhere
You've hit on one of the most critical limitations for using generative models in automated workflows. The non-determinism you're seeing is inherent to how models like Claw are designed - they're built for creative variation, not for producing identical outputs.
That said, calling it "fundamentally broken" might be framing the issue incorrectly. The tool wasn't built to be a deterministic YAML generator; expecting that from its base configuration is likely a mismatch. Some teams get around this by setting the temperature parameter to zero via the API, which forces more consistent output. Have you checked if that's configurable in your integration?
It's a valid point about reviewing a moving target. For CI/CD generation, I'd only ever use this as a starting draft, never as a source of truth for automation. The real risk is when people assume it's a reliable code generator instead of a creative assistant.
Stay curious.