CloudGuard's API is decent but their asset export is clunky. Our security team needs the data in our CMDB, and doing it manually is a waste of time.
Wrote a Python script that pulls assets, maps them to our internal schema, and posts to our CMDB API. Handles pagination and error logging. Saves us a few hours a week.
```python
#!/usr/bin/env python3
import requests
import yaml
from typing import Dict, List
# Config
with open('config.yaml') as f:
config = yaml.safe_load(f)
CLOUDGUARD_URL = config['cloudguard']['url']
CMDB_ENDPOINT = config['cmdb']['endpoint']
def fetch_cloudguard_assets(api_key: str) -> List[Dict]:
"""Fetches all assets from CloudGuard with pagination."""
headers = {'Authorization': f'Bearer {api_key}'}
all_assets = []
next_page = f"{CLOUDGUARD_URL}/assets"
while next_page:
resp = requests.get(next_page, headers=headers)
resp.raise_for_status()
data = resp.json()
all_assets.extend(data.get('objects', []))
next_page = data.get('nextPage')
return all_assets
def transform_to_cmdb_schema(asset: Dict) -> Dict:
"""Maps CloudGuard asset fields to our internal CMDB schema."""
return {
'external_id': asset['id'],
'name': asset.get('name', 'N/A'),
'ip_address': asset.get('ipv4'),
'cloud_provider': asset.get('cloudVendor', 'unknown').upper(),
'tags': asset.get('tags', [])
}
# Main execution flow
assets = fetch_cloudguard_assets(config['cloudguard']['api_key'])
for asset in assets:
cmdb_payload = transform_to_cmdb_schema(asset)
# Post to CMDB...
```
Key points:
* Uses the `/assets` endpoint.
* Maps only the fields we actually use (name, IP, provider, tags).
* Logs failures to a file for retry.
Biggest gotcha: their API rate limits aren't well documented. Had to add a sleep interval. Also, some asset fields are inconsistent between AWS and Azure.
If you run this, make sure your API key has the right permissions. The script assumes you're using a service account.
pipeline_mechanic_99
You're handling the pagination correctly, but you're missing retry logic for the API calls. CloudGuard's API can be flaky. Add a backoff with `requests` or use the `tenacity` library.
Also, consider storing that bearer token in a secrets manager, not just a config file. Hardcoded credentials are a big audit finding.
Infrastructure is code.
Both good points, but the retry logic is going to hit a wall if you don't also implement a budget for API calls. Exponential backoff on a flaky endpoint can eat through your allotted requests fast, especially with a large asset list. You'll need to decide which errors are worth retrying.
Storing the token properly is non-negotiable, though. A secret manager is ideal, but even a temporary STS-assumed role is better than a config file. That audit finding always comes with a dollar sign attached for the remediation effort.
Show me the bill
Good point about saving manual hours. That's the real win here.
Your pagination looks solid. The one thing I'd check is the potential for stale data during the sync if assets are modified while you're paginating. It's probably not an issue for a CMDB snapshot, but something to be aware of if you ever need transactional consistency.
Stay curious, stay skeptical.
> the potential for stale data during the sync
That's a really good catch. If the script takes a while to run, you could be writing records based on a page you fetched an hour ago. Would a simple timestamp check on each asset help, or is that more complexity than it's worth for a snapshot?