You've identified the exact policy complication that arises from grouping identities. The workaround I've seen is moving authorization logic into the application layer, which creates a different kind of overhead.
For the ETL scheduler example, you'd have a single Banyan device identity granting it a broad "connect" privilege. Authorization for specific read or write operations is then handled by the scheduler itself, using job metadata to query a separate internal policy engine before executing SQL. This shifts the problem from device sprawl to ensuring your orchestration layer's authorization logic is as secure as your perimeter.
It works, but now you're managing and auditing two policy systems.
every dollar counts
Yeah, that exact SQL background is where I'm coming from too. The hardest part is thinking of the service account's device as just its name, not its actual key.
So in your case, could you start with the "device per service" just for your most critical jobs, like the nightly ETL? That gives you clean logs. Then for the smaller, temporary jobs, maybe try the grouped approach user232 mentioned, just to see if the audit metadata is good enough?
How do you decide which jobs get a permanent device identity versus sharing one? Is it just about how critical they are?
Containers are magic, but I want to know how the magic works.
The device-per-service model will drown you. I've seen it with data teams.
You're used to static SQL accounts. So treat the Banyan device like your old service account name, but never let it hold the actual key. The key is a short-lived token something else fetches. The nightly ETL job is a device named `svc-nightly-report`, but the script runs on a host that gets a fresh token every few hours.
The audit trail still shows `svc-nightly-report` accessed the database. Rotation pain moves to the host identity, which you handle yearly, not daily.
Start with your 3 most critical jobs. Give them each a device. You'll see the pattern.
slow pipelines make me cranky
Start with your 3 most critical jobs, sure. Then watch as your PM demands the same audit trail for the fourth job, and the fifth. That's how the sprawl begins.
The rotation pain doesn't just "move to the host identity." You've now traded managing N service accounts for managing N device identities plus a host identity. The host becomes a single point of failure for all those jobs.
The real trap is thinking you need perfect audit trails from the infra layer. Most breaches happen because the logging was ignored, not because the job name was a metadata field.
Keep it simple
That SQL mindset is the blocker. A "device" is just a name, not a credential. Your service account `svc_etl_nightly` becomes a device record. It never holds a key.
The actual access comes from a short-lived token fetched by something else, like a Kubernetes service account on the host. The audit logs show `svc_etl_nightly` accessed the DB, but rotation is on the host identity. You manage one yearly rotation, not daily credential updates.
Start with a couple critical jobs. You'll see it's your old static account model, just with the key moved elsewhere.
show the math