We've been trying to migrate a Docker build pipeline to use GitHub Actions. It's a monorepo with a few services. When we run a matrix build for different Node.js versions (14, 16, 18), the jobs keep timing out at 6 hours.
Our workflow seems straightforward: check out, set up Node, build the Docker image, and push to ECR. It works fine for a single job, but with the matrix, we hit the limit. The logs just stop.
Is this a known limitation with the 6-hour max job duration? Are we structuring the matrix wrong, or is this just too much work for Actions? What are the typical workarounds? Splitting the workflow? Using a different runner type?
The 6-hour hard limit is indeed a documented constraint for GitHub-hosted runners. Your issue likely stems from the cumulative nature of matrix jobs, not their individual durations. The real problem is often the `docker build` step, especially with a monorepo context, as it can send the entire repository to the Docker daemon for each matrix permutation, causing massive I/O and time overhead.
You can verify this by adding a simple timestamp log before and after your build step in one job. If a single build takes, say, 2 hours, three concurrent matrix jobs will all consume that time, but the 6-hour clock is per job. The workaround isn't splitting the workflow, but optimizing the build context. Use a `.dockerignore` file aggressively and consider a multi-stage build that copies only necessary artifacts. Alternatively, pre-build dependencies in a separate, cached job.
Have you profiled which stage consumes the most time? The timeout could be masking a different bottleneck, like network latency during the ECR push.
numbers don't lie
The 6-hour limit is per job, so your matrix setup likely isn't the direct cause. The more probable culprit is that each parallel job is independently hitting the timeout due to the build context size. A single job finishing doesn't guarantee the matrix variants will.
Have you examined the logs from a failed matrix run to see where exactly the time is spent? It's often layer caching or the `COPY` step. You might need to implement buildkit caching or use a separate, smaller context directory for your Docker build instead of the whole monorepo root.
prove it with data
Yeah, that 6-hour limit can really sneak up on you. The other replies are spot on about the build context, but I've found another layer to this: the default GitHub-hosted runner storage.
If your monorepo is large, even with a good .dockerignore, you're still pulling the whole thing for each matrix job. Each runner gets its own storage, so you're multiplying that download and the Docker context send for every Node version. I ran into this building for multiple architectures.
One trick that worked for me was using a setup job to pre-build a tarball of just the needed context and upload it as an artifact, then have the matrix jobs download that. Cuts down the repetitive heavy lifting. Also, are you using the standard `ubuntu-latest` runner? The disk space there can get choked.
Automate all the things.