What's the best way to compare build times between GitLab CI and CircleCI?

Hi everyone,

I've been migrating some of our team's pipelines from GitLab CI to CircleCI (and sometimes vice versa) and the most common question from management is always about performance: "Which one is faster for our use case?" I've found that comparing build times isn't just about running the same `.gitlab-ci.yml` and `.circleci/config.yml` and seeing which number is lower. You need a structured approach to get meaningful, actionable results.

Here's the method I've settled on for a fair comparison:

**1. Isolate the Variables You Can Control**
- Use the same codebase, commit hash, and dependencies in both systems.
- Standardize the compute: match runner specs (vCPUs, memory) as closely as possible. For GitLab, you might use `tags` on a specific runner; for CircleCI, define it in `resource_class`.
- Cache strategy is huge. You must implement equivalent caching for both (e.g., Docker layer cache, dependency cache). An unfair cache advantage will skew everything.

**2. Define Your Benchmark Pipeline**
Break down a typical pipeline into stages and compare each stage individually. A aggregate "total time" can hide important differences. I usually track:
- Dependency installation (npm/pip/maven)
- Linting/formatting
- Build/compilation
- Unit tests
- Integration tests
- Container build/push

**3. Use Measurable Metrics & Tools**
Don't just look at the CI tool's UI timers. Instrument your pipeline to log timestamps. A simple script can help:

```bash
#!/bin/bash
START_TIME=$(date +%s)
# Your build step here
END_TIME=$(date +%s)
ELAPSED_TIME=$((END_TIME - START_TIME))
echo "STEP_TIME=$ELAPSED_TIME" >> $METRICS_FILE
```

Then, collect these metrics over, say, 10-20 runs for each CI system to account for network variability.

**4. Account for the "Hidden" Factors**
- Queue time: How long does a job wait for an available runner? This can differ drastically between your self-hosted GitLab runners and CircleCI's cloud.
- Setup/teardown overhead: The time to provision a clean environment or clone the repo.
- Network latency to your artifact storage (e.g., S3) or container registry.

My personal finding has been that for medium-sized monorepos, GitLab CI with well-tuned self-hosted runners often wins on raw execution time because you control the hardware. But CircleCI can be more consistent and faster on the queue time if you're relying on their infrastructure. The "best" truly depends on your team's willingness to manage runners and your cost tolerance.

Has anyone else run a similar comparison? I'm particularly curious about how you've benchmarked caching performance between the two platforms.

-- Amy