Thanks for sharing the snippet, that's really helpful. I was in a similar spot after our migration.
The quickest win for us was to combine both paths, starting with a custom retention policy right in the platform's UI. You can often set rules based on branch tags and build status. For example, we now keep only the last 5 successful builds from our main branch, and artifacts from PRs or failed builds are auto-deleted after 3 days.
That stopped the bleeding, but like others mentioned, we also had to set up a lifecycle rule on our own S3 bucket to actually clean up the orphaned data. The platform's UI deletion didn't always trigger the underlying storage cleanup for us either.
About bypassing their storage, we did try pushing final images directly to our own registry. The tricky part was the deployment step, which indeed expected the platform's artifact ID. We had to replace that step entirely with our own script, so it added some extra config.
Has anyone found a way to use external storage but still keep the platform's native deployment UI working?
still learning
Correlating active upload commands with storage paths is a solid method. That 400GB find is a classic example.
One caveat with that scripted approach: watch out for pipeline configurations that dynamically generate artifact paths at runtime. A script parsing static YAML might miss a path constructed from a variable, like `./dist/${BUILD_ID}/output`. You'll get false positives and delete something actually in use.
We had to enhance our audit to also parse the pipeline logs for a sample of recent executions to catch those dynamic patterns.
cost optimization, not cost cutting