During our recent migration from Jenkins to GitLab CI, we encountered an unexpected constraint: a hard limit on the size of the `.gitlab-ci.yml` file. Our initial translation effort produced a monolithic pipeline definition that exceeded the platform's 1 MB limit for the file.
The primary cause was our attempt to directly map dozens of complex, parameterized Jenkins library functions into a single YAML file. This included extensive inline scripts and conditionals for various deployment environments. The migration wasn't simply a syntax translation; it required a fundamental restructuring of our pipeline logic.
We resolved the issue by:
* Decomposing the monolithic file into smaller, reusable templates using the `include:` keyword.
* Moving lengthy shell scripts into separate repository files and calling them.
* Leveraging GitLab CI's `rules:` and `workflow:` keywords to replace complex conditional logic that was previously expressed in YAML.
This experience highlighted that migration is as much about adapting to the new platform's philosophy as it is about syntax. A direct one-to-one translation of pipeline logic can lead to hitting operational limits. The refactoring added approximately two days to our timeline, but resulted in a more maintainable and modular pipeline structure.
Has anyone else encountered similar platform-specific limits—be it file size, nesting depth, or concurrent job constraints—during a migration? I'm particularly interested in how different platforms handle complexity scaling.
—lucas
That's a good point about platform philosophy. I've seen similar "lift and shift" mentality blow cloud budgets - translating an on-prem VM catalog directly to EC2 instances often hits service quota limits, not to mention the cost shock.
Your fix with `include:` templates reminds me of breaking down a massive CloudFormation template into nested stacks. The initial single-file approach feels easier until you hit a hard ceiling. Did you find the refactoring also made cost attribution easier? Like, could you tag or isolate pipeline runs per team after splitting things up?
Interesting. That 1 MB limit is surprisingly small for a complex pipeline. Did the platform give any warning before failing, or was it just a hard stop during a push?
I'm curious about the trade-off. Breaking things into templates seems cleaner, but doesn't it add complexity in tracking down where a job is actually defined?
That 1MB limit is one of those platform-specific constraints that's easy to miss until you hit it. Did you find that the refactoring process using `include:` actually exposed other logical groupings you hadn't considered before? Sometimes breaking for a technical limit forces a better architectural split.
I'm glad you mentioned adapting to the platform's philosophy. It's a common migration trap. You see a similar thing with moving from CloudBees Jenkins to something like GitHub Actions - the pattern for reusable logic shifts from shared libraries to actions or composite run steps, and trying to force the old pattern just doesn't scale.
Mike D.
Absolutely. The forced decomposition did reveal logical groupings, primarily around cost centers and change velocity. Our original Jenkins file was organized by pipeline stage (build, test, deploy). The refactoring pushed us to organize by team ownership instead.
For example, we created separate template files for:
* Security scanning - owned and updated by the infosec team
* Compliance artifact generation - owned by legal/audit
* Performance testing - owned by the platform SREs
This mirrored our FinOps tagging strategy and made it easier to attribute pipeline runtime costs back to those teams. The technical limit created a pressure that aligned the CI/CD structure with our organizational and financial governance.
Your parallel to GitHub Actions is spot on. The mental shift from a centralized library to distributed, versioned components is non-trivial. Teams that don't make that conceptual leap end up with a tangled web of includes that's worse than the original monolith.
Trust but verify. Then renegotiate.
That shift to organizing by team ownership instead of pipeline stages is really interesting. Did you run into any pushback from teams who suddenly had to maintain their own CI templates? Seems like it could add overhead for them.
Still learning.
That's such a crucial example of a technical constraint forcing a better, long-term architectural decision. It's not just about making the file smaller to fit, but about how the refactoring changes the way you think about pipeline ownership.
I've seen similar patterns in CRM migration, where trying to translate a legacy system's massive, tangled automation scripts directly into the new platform's workflow engine hits governor limits. The answer is always decomposition, but the *logic* you use to decompose - by team, by cost center, by change velocity - ends up being more valuable than just getting under the limit. It creates cleaner boundaries for testing and rollback plans.
Your point about moving scripts out of YAML is key. It also makes versioning and code review for those scripts themselves much more manageable, as they can live in their own files with proper linting and testing.
test the migration before you migrate