Skip to content
Notifications
Clear all

Did you see the post about CI cost leaks from large artifact storage?

2 Posts
2 Users
0 Reactions
2 Views
(@code_reviewer_anna_v2)
Estimable Member
Joined: 4 months ago
Posts: 129
Topic starter   [#21560]

Hey folks! 👋 I was just reading that deep dive on CI/CD storage costs, and it really hit home. The article mentioned teams getting "leaked" hundreds per month just from old artifacts sitting in cloud storage. It's so easy to forget that every `node_modules` tarball and docker layer cache adds up.

I recently audited our own pipelines and found a huge culprit: we were using a generic "retain 30 days" policy, but some build jobs were generating 2GB+ artifacts *each run*. Our monthly storage bill was almost doubling because of it! Here's the simple retention rule we added to our GitHub Actions workflow that saved us ~40%:

```yaml
- name: Cleanup Artifacts
run: |
# Delete artifacts older than 7 days, keep latest 5 regardless of age
gh api repos/${{ github.repository }}/actions/artifacts --paginate
| jq -r '.artifacts[] | select(.created_at < "'$(date -d '-7 days' '+%Y-%m-%d')'") | "(.id)"'
| tail -n +6 | xargs -I {} gh api -X DELETE repos/${{ github.repository }}/actions/artifacts/{}
```

For self-hosted runners, the cost is more about the disk space management. We set up a cron job to purge old files from the runner's workspace:

```bash
find /path/to/runner/_work -name "*.tar.gz" -mtime +5 -delete
find /path/to/runner/_work -type d -name "node_modules" -mtime +3 -exec rm -rf {} +
```

Some best practices we learned:
* **Audit artifact generation:** Do you *really* need to store that build output, or can it be reproduced?
* **Implement tiered retention:** Keep debug builds for 2 days, release builds for 30.
* **Use caching aggressively:** A well-configured cache (e.g., for dependencies) often reduces artifact size dramatically.
* **Monitor growth:** Set up alerts when storage usage spikes.

Has anyone else done a similar cost audit? I'm curious what other "leaks" people have foundβ€”maybe in long-lived runner instances or inefficient caching strategies.

Happy coding!


Clean code, happy life


   
Quote
(@bench_beast)
Reputable Member
Joined: 1 month ago
Posts: 232
 

That jq date comparison is brittle if your artifacts have timestamps with different formats. Better to use `date -d '-7 days' '+%Y-%m-%dT%H:%M:%SZ'` for full ISO string matching.

For runners, the disk space gets eaten by Docker more than workspace. We added a `docker system prune -af --volumes` to the cron job.


Benchmarks don't lie.


   
ReplyQuote