Skip to content
Notifications
Clear all

Anyone else seeing slow data transfer during Jenkins to GitHub Actions migration?

2 Posts
2 Users
0 Reactions
4 Views
(@tool_skeptic_45)
Eminent Member
Joined: 5 months ago
Posts: 14
Topic starter   [#60]

We're migrating 200+ Jenkins pipelines to GitHub Actions. The job conversion itself is straightforward, but the data transfer is a bottleneck.

We're using the GitHub Actions Importer for the bulk of it. Our problem: transferring historical build logs and artifacts. The API rate limits are killing us, even with PATs and queued jobs.

Current setup:
```yaml
# Using the CLI for migration
gh actions-importer migrate jenkins
--source-url https://our-jenkins.internal
--config-file_path config.yml
```

Key issues:
* Transferring 2TB of artifact data. GitHub's API/sync isn't built for this volume.
* Cutover plan assumes data moves fast, but we're looking at weeks at current speeds.
* Historical log preservation is becoming a "nice to have" we might drop.

Questions:
* Did you hit similar data transfer walls?
* Did you just cut over without history and keep Jenkins read-only for a year?
* Any workarounds besides "pay for more concurrent jobs" (we already are)?
* Is this a sign we're trying to force a square peg into a round hole?



   
Quote
(@Anonymous 315)
Joined: 1 week ago
Posts: 7
 

The 2TB artifact transfer is the real problem. The GitHub Actions Importer tool is for pipeline logic, not bulk data. You're hitting fundamental API constraints.

We faced this and chose a hybrid archival strategy. We kept all artifacts on S3, pointed a metadata-only migration at them, and kept Jenkins decommissioned but reachable for a year via a read-only proxy. Historical logs were converted to a compressed, queryable format (like Parquet) and stored cheaply, not in GH. This cut migration time to days.

It's not a square peg, but you can't lift-and-shift the data layer. Treat the migration as a chance to separate your durable artifact store from your CI/CD orchestrator. Could you use this to implement a proper artifact retention policy instead?



   
ReplyQuote