We're about 18 months into running GitLab CI/CD for our core product development. Org size is around 500, with about 120 active developers/engineers committing to the main mono-repo. I was brought in during the vendor evaluation phase, and we ultimately went with GitLab over a Jenkins/ArgoCD combo and GitHub Actions.
The short answer is yes, it's running in production and it's stable. But the value proposition changes dramatically based on how you license and structure it.
Our setup:
* Self-managed GitLab Ultimate on our own K8s cluster.
* ~4500 pipeline jobs per day on average.
* Mono-repo with forced trunk-based development (no long-lived feature branches).
The good:
* The single pane of glass is real. Having issues, MRs, CI, and security scans in one place reduced a lot of context switching for teams. Our security compliance folks love the baked-in license scanning and SAST reports.
* The built-in container registry and package registry are workhorses. We eliminated two separate vendor tools by using these.
* Pipeline configuration is straightforward for most patterns. When it works, it's very maintainable.
The gotchas (where the war stories come in):
* **Cost at scale:** The runner concurrency is the killer. We hit our licensed limit quickly. The negotiation for more concurrent runners was... intense. You need to model your peak pipeline load very carefully. We now use autoscaling on AWS EC2 for our runners, but the license tier gates the concurrency.
* **Complex pipelines need careful design:** We had a "mega-pipeline" that grew organically and started timing out. We refactored into parent-child pipelines using `trigger:` and `include:`. Performance improved, but the complexity shifted to pipeline management. Debugging a downstream child pipeline failure can be a chore.
* **The "vendor lock-in" feeling:** While you can export data, moving your entire CI/CD configuration and history to another platform would be a major project. This isn't unique to GitLab, but it's a consideration. We mitigated this by strictly keeping build scripts (e.g., `build.sh`) in the repo, separate from the `.gitlab-ci.yml` logic.
Performance-wise, for a typical merge request pipeline (build, unit test, security scan):
* Average time from push to pipeline start: < 60 seconds (depends on runner availability).
* Average pipeline duration: 9-11 minutes.
* Main branch pipeline (full integration + e2e): ~35 minutes.
Would I recommend it? For a 500-user org already on GitLab for source control, yes, but with caveats:
* Get a clear understanding of your pipeline concurrency needs before you sign.
* Budget for a dedicated person to own and evolve the pipeline patterns. It's not a "set and forget" system at our scale.
* The value is highest if you use the other Ultimate features (security, compliance, portfolio management). If you just want CI, it might be overkill.
I'm curious if others have hit similar scaling walls or found clever ways to optimize runner usage or pipeline structure.