Our JFrog Xray instance keeps running out of disk space because the database cleanup job (`jfrog xray db-cleanup`) fails. The error in the logs is generic: "Cleanup process failed." Storage is full, so the job can't proceed.
We've tried:
* Increasing the disk space (temporary fix, fills up again).
* Running the command manually from the Xray server.
* Checking PostgreSQL logs—no obvious errors there.
Current config (relevant part from `system.yaml`):
```yaml
shared:
database:
postgresql:
maxOpenConnections: 20
```
What specific metrics or logs should we check to force this cleanup? Is there a way to target specific old data for deletion first to free up space for the job to run? Need concrete steps.
Benchmarks don't lie.
The generic error suggests the cleanup job is hitting a preflight disk space check before even starting transactional operations. You need to look at the Xray service logs, not just PostgreSQL. Check `$JFROG_HOME/xray/var/log/console.log` for the exact minute you ran the job - it should log the specific cleanup phases and which table it fails on.
Since you can't run the full job, you'll have to manually free space by targeting the largest historical data tables first. Connect directly to the Xray PostgreSQL database and run this to identify candidates:
```sql
SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) as total_size
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC
LIMIT 10;
```
Typically, `events`, `violations`, and `component_versions` hold the most historical data. You can create a one-off delete for records older than a certain date (e.g., older than 180 days) for the largest table, but you must do it in small batches to avoid long-running transactions. Use a `DELETE FROM events WHERE created < '2023-01-01' LIMIT 10000;` and repeat until you've freed enough space for the automated job to take over.
Also verify your `system.yaml` has the cleanup retention settings configured - if they're absent, the job might be trying to delete nothing while still requiring working space for its vacuum operations.
data is the product