Having recently undertaken the migration of our Aqua Security deployment from their managed SaaS offering to a self-hosted, on-premises Kubernetes environment, I feel compelled to share a critical and substantially underestimated aspect of this process: the profound shift in storage architecture and its operational burden. While the documentation covers the procedural steps for the migration itself, the ongoing storage implications for a self-hosted setup are, in my experience, inadequately highlighted and have introduced significant overhead.
The core issue resides in the database requirements for the self-hosted Aqua Server. In the SaaS model, the storage layer is, naturally, abstracted away. When you transition to self-hosted, you are responsible for provisioning and maintaining a PostgreSQL database of considerable scale. The sizing guidelines provided are deceptively simple, and our real-world experience has shown that the growth trajectory of the `aqua_db` is neither linear nor easily predictable.
Key pain points we've encountered include:
* **Volume Growth Rate:** The primary tables related to scans, vulnerabilities, and audit events exhibit aggressive growth. Without a meticulously planned and aggressive data retention policy—which must be manually configured and monitored outside of the Aqua console—the database can expand at a rate of tens of gigabytes per month in an environment with moderate to high scanning activity.
* **Performance Tuning:** The default PostgreSQL parameters are insufficient for production loads. We had to engage in non-trivial performance tuning, particularly around connection pooling, autovacuum settings, and work memory, to prevent transaction lock waits and query timeouts during peak scanning periods. This requires dedicated DBA attention, a resource we did not anticipate needing for this application.
* **Backup and Recovery Complexity:** Implementing a robust backup strategy for this database is now our responsibility. Point-in-time recovery (PITR) is essential for a security tool, necessitating continuous WAL archiving. The storage footprint for backups alone quickly became a separate storage planning exercise, effectively doubling our initial allocation estimates.
* **Operational Overhead:** Monitoring database health, planning for vertical scaling events, and managing storage tiering (to balance performance for recent data against cost for archived data) have become ongoing tasks. This is a stark contrast to the SaaS model where scaling was seamless and invisible.
My question to the community, and the reason for this post, is to inquire whether others have navigated this transition and what strategies have proven effective for managing this storage layer. Specifically:
* Have you implemented successful data archival or aggregation strategies outside of Aqua's built-in settings to curb database growth?
* Are there specific PostgreSQL extensions or table partitioning schemes you have employed to improve manageability?
* Does anyone utilize external object storage or cloud database services for this component in a hybrid model, and if so, what are the latency implications?
The functional benefits of self-hosting Aqua are clear for our compliance requirements, but the storage and database management burden has been the most substantial and unexpected cost, both in terms of infrastructure and ongoing operational effort. It is a crucial consideration for any team contemplating a similar move.
You're hitting on the biggest hidden cost, and it's not just the raw storage capacity. The aggressive growth of those tables creates a secondary problem: performance degradation on even moderately sized Postgres instances if you don't get your indexing and vacuum strategy absolutely right from day one. We saw query times for the console spike after a few months because autovacuum couldn't keep up with the churn in the scan results tables.
Did you also run into issues with the required `ENHANCED` monitoring database? Sizing that for high-verbosity audit logging felt like provisioning for a separate application entirely. The default retention policies aren't nearly aggressive enough for a self-hosted operator thinking in terms of actual storage costs.
Design for failure.
Absolutely spot on about the sizing guidelines being deceptively simple. They often start from a "per 1000 images" or "per scanner" baseline that doesn't account for the *churn* in a real CI/CD pipeline, where you might be scanning the same number of images, but with new layers and new vulnerabilities each time, leading to exponential row growth in those event tables.
Your point on unpredictable growth reminds me of a specific trap: the default configuration for retaining scan history. If you don't tune the `AUDIT_RETENTION_PERIOD` and `SCAN_HISTORY_RETENTION` environment variables aggressively from day one, you're essentially provisioning for a data warehouse, not an operational database. We learned the hard way and had to implement a separate, pre-deployment cron job in our Helm chart to prune old data, because the built-in cleanup wasn't keeping up.
Have you considered offloading historical data to a separate, cheaper object storage and using a tool like TimescaleDB for the fresh data? It's more operational complexity, but it changes the cost curve from a vertical line to something more manageable.
Prod is the only environment that matters.
You've zeroed in on the exact operational pivot that most teams fail to budget for, both in terms of storage cost and administrative toil. The abstraction of storage in the SaaS model isn't just a convenience; it's a complete removal of a data engineering problem that you now inherit.
Beyond provisioning for growth, you need to immediately implement a data lifecycle strategy the SaaS provider handled silently. This includes partitioning schemes for the scan results tables, aggressive retention policies that are stricter than defaults, and a dedicated maintenance window for aggressive `VACUUM` operations on the high-churn tables. Without this, performance will degrade within weeks, not months.
Consider treating the `aqua_db` not as a traditional application database but as a high-velocity time-series data store. This mental shift forces the architectural decisions - like automated roll-offs and tiered storage - that are necessary for control.
—KM
You're so right about that mental shift. Thinking of it as a high-velocity data store is key, it changes your whole approach to retention and queries.
One thing we had to learn the hard way: you also need to bake your maintenance strategy into your monitoring from day one. Just having a cron job for VACUUM isn't enough. We set up alerts for table bloat and scan history growth rate, otherwise you're still reacting when it's too late.
That admin toil is real - what felt like a one-time migration project turned into a permanent, part-time DBA role for someone on the team.
Automate the boring stuff.
The aggressive growth in scan and vulnerability tables is something you can't model from the documentation alone. We found the ingestion rate for a single CI/CD scanner was an order of magnitude higher than projected because the data model stores a discrete record per finding per image layer per scan. This isn't just about raw gigabytes, it's about rows per second during active scanning windows, which directly dictates your required IOPS and forces an immediate move to provisioned storage tiers on cloud PostgreSQL.
Our operational breakthrough was implementing time-based table partitioning on the `scan_results` table from the initial deployment, which the Helm chart does not configure. It turns what would be a catastrophic `DELETE` operation for retention enforcement into a near-instant `DROP TABLE`. Without that partitioning scheme, even a tuned autovacuum can't salvage performance.