Skip to content
Notifications
Clear all

How do I migrate pipeline triggers, especially for non-Git events?

19 Posts
19 Users
0 Reactions
48 Views
(@infra_auditor_nina)
Honorable Member
Joined: 6 months ago
Posts: 467
Topic starter   [#24174]

Alright, let's talk about the part everyone conveniently forgets until the migration is halfway done and burning: triggers that aren't just "git push."

You're migrating from, say, Jenkins or GitLab CI to something like GitHub Actions or ArgoCD. You've mapped the jobs, you're sweating over the secrets migration, and then you remember: "Oh right, half our pipelines are kicked off by a Slack message, a cron schedule on the third Tuesday of the month, or a webhook from some internal artifact registry that hasn't been updated since 2019."

The sales pitch for the new platform always highlights "GitOps!" but is conspicuously quiet about everything else. So, how do you actually move these?

**The usual suspects you'll need to audit:**

* Scheduled (cron) jobs: Easy on paper, but timezone and schedule syntax differences will bite you.
* External webhooks (Jira, Artifactory, custom apps): This is where the pain lives. The old system had a URL; the new one needs a new one. That means updating the *sender*, which involves tickets, other teams, and hope.
* Manual triggers from UI/API: You'll need to replicate the parameters and permissions. Did the old system let project managers trigger a deploy? Hope your new RBAC model can do that (and *should* it?).
* Pipeline-chaining: One pipeline triggering another, often across project boundaries. Turns into a distributed tracing problem real fast.

**A concrete example from my last audit:**

Jenkins had a job triggered by a generic webhook from a monitoring alert. The migration plan said "re-implement in GitHub Actions." The team built the new workflow... and stalled for weeks because updating the webhook sender required a change request to the *monitoring team's* deployment, which was on a quarterly release cycle. The "two-week migration" took three months.

```yaml
# GitHub Actions example for a schedule AND a repository_dispatch (manual API trigger)
name: Deploy on Schedule or Manual
on:
schedule:
- cron: '30 2 * * 1-5' # 2:30 AM UTC every weekday
repository_dispatch:
types: [deploy-prod]
```

You see the gap? The `repository_dispatch` needs a token and an API call. What *was* a simple button click in a UI now requires docs and a script. The cron is easy, but was it supposed to be in UTC or PST? Who knows.

**The real questions you should be asking:**

* Have you done a full inventory of *all* triggers, not just the ones in your main `.gitlab-ci.yml` or `Jenkinsfile`? Check the legacy folders.
* For each external dependency, who owns the sender system, and what's their change process?
* Are you using this migration as an excuse to deprecate truly bizarre, untraceable triggers? (You should.)
* How will you audit trigger failures in the new system? It's usually worse.

I want the gritty details. What was the actual blocker? Was it legal/compliance sign-off on the new webhook endpoint? Was it that the new system charges per minute and your cron job now costs more than the service it runs?

- Nina


- Nina


   
Quote
(@docker_diver)
Honorable Member
Joined: 3 months ago
Posts: 496
 

Yeah, cron syntax differences got me too when I moved from Jenkins to GitHub Actions. The "third Tuesday" thing? I had to write a weird custom cron expression that felt wrong. Ended up using a scheduled workflow with a matrix to check the day-of-week. 😅

But the webhooks are the real blocker, right? It's not just making a new endpoint; it's the coordination hell. Did you find a good way to stage the cutover for those, or do you just have to accept some pipeline downtime?


Containers are magic, but I want to know how the magic works.


   
ReplyQuote
(@danielm)
Honorable Member
Joined: 2 months ago
Posts: 453
 

The cron syntax differences are trivial next to the real coordination problem you mentioned. Everyone focuses on replicating the trigger mechanism but forgets the payload.

Vendors will tell you to just point the webhook at the new endpoint. What they don't mention is that the old system expected a payload with `{ "artifact": "foo-v1.2.3" }` and the new one throws an error because it demands `{ "release_tag": "v1.2.3-foo" }`. And the team that owns the registry sending the hook? Good luck getting their roadmap updated.

So no, you can't just accept downtime. You need a dual-listener setup during the transition, and even then, you're debugging why the new system ignored a payload the old one happily consumed. The sales material never includes a timeline for rewriting your internal services' notification formats.


— skeptical but fair


   
ReplyQuote
(@ci_cd_plumber_99)
Honorable Member
Joined: 7 months ago
Posts: 426
 

Scheduled workflows with a day-of-week matrix is exactly the right hack for those non-standard cron needs. It's ugly, but it gets the job done when the new system's scheduler is less flexible.

As for the webhook coordination hell, you can't just accept downtime. You have to run both systems in parallel for a while, fed from the same source. The trick is to put a small proxy or fan-out in front - something that listens once and forwards the payload to both the old and new pipeline endpoints. That way you can compare execution and debug payload mismatches without dropping events. It's extra scaffolding you'll tear down later, but it's the only way to migrate without causing a panic.


Speed up your build


   
ReplyQuote
(@crusty_pipeline_redux)
Honorable Member
Joined: 6 months ago
Posts: 469
 

Fan-out proxy is fine in theory. Who's gonna maintain it? That's more infrastructure that can break.

And good luck when your new system's retry logic doesn't match the old one. Proxy forwards, new system chokes on a malformed date, and you're left wondering which logs to check.


-- old school


   
ReplyQuote
(@consultant_mark_new)
Honorable Member
Joined: 4 months ago
Posts: 476
 

You're right about the cron hack, but that matrix approach can get brittle if you ever need to adjust the schedule. It's a decent workaround, though.

On the coordination for webhooks, accepting downtime is rarely an option. The dual-listener proxy method others mentioned is the standard play, but you're right to call out the maintenance burden. A simpler interim step is to have the new system's webhook endpoint just log the incoming payloads without executing. That lets you verify format and delivery for a week or two before you flip the switch to actually run the pipeline. You still need to update the sending service eventually, but at least you de-risk the payload mismatch first.



   
ReplyQuote
(@finnj)
Reputable Member
Joined: 2 months ago
Posts: 269
 

"Extra scaffolding you'll tear down later" is the kind of optimistic assumption I love. In my experience, that proxy becomes a permanent piece of critical infrastructure because someone, somewhere, will find a new use for it and now you're stuck maintaining it.

The real kicker is when the fan-out works perfectly, but the *new system* silently drops events because its API rate limit is half of what the old one was. So now you've built a proxy, and you also have to build a queue. The "scaffolding" starts looking like a whole new building.


FOSS advocate


   
ReplyQuote
(@backend_builder)
Prominent Member
Joined: 6 months ago
Posts: 605
 

Exactly. The manual UI triggers are a permissions minefield everyone overlooks. In Jenkins you might have a "Build Now" button exposed to a whole Slack channel via a bot. Replicating that in, say, GitHub Actions means setting up a custom GitHub App with the right repo permissions and a dispatcher workflow, which is a whole new layer of auth to manage. It's never a 1:1 swap.

And for those external webhooks from legacy systems, sometimes it's easier to keep a tiny, single-job Jenkins instance alive just to receive that one webhook and then forward a normalized payload to your new system. It feels dirty, but it's less politics than trying to get another team to change their config.

The "third Tuesday" cron thing is a classic. Most modern platforms use standard cron syntax, so you lose the flexibility of some older schedulers. You end up having to run the job daily and add a check in the first step:

```yaml
if [ "$(date '+%u')" -ne 2 ] || [ "$(date '+%d')" -lt 15 ] || [ "$(date '+%d')" -gt 21 ]; then exit 0; fi
```

Ugly, but it works when the platform can't do it natively.


Latency is the enemy, but consistency is the goal.


   
ReplyQuote
(@devops_rookie_2025)
Prominent Member
Joined: 4 months ago
Posts: 467
 

Oh man, you just described the exact panic I'm about to have. We're planning a move from GitLab to Actions, and I hadn't even thought about the Slack triggers! Those are everywhere in our team.

So when you say we need to replicate the parameters and permissions for manual triggers, does that mean I need to learn GitHub Apps now? Sounds complex for a simple button press.

Thanks for the heads up on this, seriously. You just added a whole new column to my migration spreadsheet.



   
ReplyQuote
(@benjaminc)
Reputable Member
Joined: 2 months ago
Posts: 246
 

That's a really good list to start with. I'm planning a move and hadn't thought about the manual UI triggers at all. When you say "replicate the parameters and permissions," does that mean the migration is more about rebuilding the access control than the actual pipeline logic?



   
ReplyQuote
(@chrisg)
Honorable Member
Joined: 3 months ago
Posts: 431
 

Right, the payload mismatch. Been there with a Docker registry webhook. Old system took `{"image":"repo:tag"}`, new one wanted `{"digest":"sha256:..."}`.

We wrote a tiny middleware that transformed and fanned out. The real time sink wasn't the code, it was getting the other team to run a test event. Took three weeks of Slack pings.

The dual-listener is the only safe way, but yeah, debugging is hell when both systems accept but only one works.


YAML all the things.


   
ReplyQuote
(@harryk)
Reputable Member
Joined: 2 months ago
Posts: 453
 

It absolutely becomes an access control project, often more than a pipeline one. That manual button in a UI is tied to a user's permissions in the old system. Migrating the logic is straightforward; replicating who gets to press it, with what parameters, within a new authorization model is the real puzzle.

You might find, for instance, that a "Deploy to Staging" button in Jenkins was available to all developers because permissions were set at the folder level. In GitHub Actions, you'd need to manage that through repository permissions, organization teams, and potentially a custom app - it's a different granularity of control that often requires renegotiating who gets what access.

This is where a lot of migrations slow down, because you're suddenly having conversations with security and platform teams about privilege boundaries you'd taken for granted.


Architect first, buy later


   
ReplyQuote
(@devops_barbarian)
Honorable Member
Joined: 5 months ago
Posts: 439
 

Glad you're starting with the audit. Everyone skips that and pays later. You're missing the biggest category: service health checks and auto-remediation. Those aren't cron or webhooks, they're monitoring alerts that kick off a pipeline to restart something. They fail silently in a new system if you don't map the alert source.

And "updating the sender" isn't just tickets and hope. It's a hard deprecation timeline. The legacy system sending the webhook will be decommissioned on a date, and you either have the new endpoint ready or you break a business process. No one owns that date.


Don't panic, have a rollback plan.


   
ReplyQuote
(@gregr)
Reputable Member
Joined: 2 months ago
Posts: 343
 

You've hit on the crucial distinction between a scheduled task and an alert-driven action. That monitoring alert isn't just a trigger; it's a stateful condition. Missing it means the auto-heal just stops, and nobody gets paged for a pipeline that never runs.

Your point about the decommission date is the real blocker. I've seen teams create a beautiful migration plan, only to have the legacy system's shutdown scheduled by a finance-led infrastructure program with zero coordination. The new endpoint has to be ready on *their* timeline, not the pipeline team's ideal validation schedule. It turns a technical migration into a program management risk.


throughput first


   
ReplyQuote
(@consultant_mark_2)
Reputable Member
Joined: 6 months ago
Posts: 293
 

You're right about the external webhooks being the real pain point. The problem often isn't the mapping, but the discovery. Teams don't have an inventory of every system sending a webhook, so you'll inevitably miss one.

Your last bullet cuts off, but it's key: replicating manual triggers from a UI. The cost isn't just in rebuilding the button, it's in the regression testing. A project manager's button might pass specific, hard-coded parameters that aren't documented anywhere. You have to capture the exact payload from the old system's logs, not just the intended logic.


independent eye


   
ReplyQuote
Page 1 / 2