Skip to content
Notifications
Clear all

Switched to self-hosted runners on GitHub Actions, storage costs bit us

3 Posts
3 Users
0 Reactions
4 Views
(@devops_dad_joke_v3)
Estimable Member
Joined: 3 months ago
Posts: 103
Topic starter   [#6330]

Switched to self-hosted runners to save on those sweet, sweet compute minutes. Thought we were geniuses.

Turns out, our artifact retention policy was set to "keep everything forever." Who knew a month of iOS builds could cost more than my mortgage? The storage bill was the real runner-up. 😑

Check your `actions/upload-artifact` retention settings, folks. The default is 90 days, but if you're on a roll and override it... you might be paying for nostalgia.


Deploy with love


   
Quote
(@consulting_contractor_mike)
Estimable Member
Joined: 4 months ago
Posts: 123
 

I'm a consulting cloud architect, currently embedded at a mid-size fintech (350 devs) that migrated from GitHub-hosted runners to a hybrid self-hosted model last year to handle our high-frequency CI for containerized services and mobile apps, including the exact iOS pipeline pain you described.

1. **Runner scaling and cost visibility** - Self-hosted runners have zero per-minute compute cost, but infrastructure and storage become your direct responsibility; we saw artifact storage on Azure Blob hit $1,200 monthly before policy fixes, compared to the GitHub Actions included storage pool. The cost shift is from transparent consumption to capital and operational overhead.
2. **Retention policy granularity** - GitHub-hosted artifact retention is a global or per-repository setting with a 90-day default; self-hosted artifact storage delegates policy entirely to your object storage lifecycle rules, which we configured via Terraform to auto-tier after 30 days and delete after 90, a manual setup that's easy to overlook.
3. **Maintenance and security burden** - Self-hosted runners require weekly maintenance (image updates, security patches, runner version upgrades); our team spends about 8-10 engineering hours monthly on runner upkeep, whereas hosted runners are zero-touch. You also inherit hardening the runner images against compromise.
4. **Performance predictability** - Self-hosted runners on our Azure VMs (Standard_D8ds_v4) provided consistent build times, about 40% faster for our iOS builds due to persistent derived data caches, but this required implementing our own 500GB SSD cache layer, adding another $80/month per runner pool.

I'd recommend sticking with self-hosted runners for high-volume, predictable workloads like your iOS builds, but only if you implement infrastructure-as-code for the storage lifecycle from day one. To make a clean call, tell us your monthly build volume in minutes and whether your team has dedicated DevOps capacity to manage the runner fleet.


Mike


   
ReplyQuote
(@ericd)
Reputable Member
Joined: 1 week ago
Posts: 180
 

That third point about the **maintenance and security burden** is the real sleeper hit. You mentioned "weekly maintenance" taking up about 8-1? I'm assuming 8-10 hours?

It's not just the patching and updates, it's the unpredictability. You get a zero-day announcement and suddenly your whole CI/CD is frozen until you can patch and redeploy your runner images. That operational tax is easy to underestimate when you're only looking at the compute minute savings. Teams forget they're now running a mini cloud provider for their builds.

We also found that without strict hygiene, runner groups can sprawl, leading to inconsistent environments and security drift. It's a trade-off for sure.


Keep it civil, keep it real.


   
ReplyQuote