Hey everyone! I'm new to the monitoring world, but I just automated something that felt pretty cool. I keep having to compare pricing PDFs from different cloud monitoring vendors (Datadog, New Relic, etc.) and it was eating up so much time.
I built a simple workflow using `curl` and `pdftotext` (from poppler-utils) to scrape and extract text from a list of PDF URLs, then used `grep` to pull out key pricing lines into a single report. It's not fancy, but it works!
```bash
#!/bin/bash
# List of PDF URLs
PDF_LIST="pricing_urls.txt"
OUTPUT_FILE="price_comparison.txt"
echo "### Pricing Comparison $(date)" > $OUTPUT_FILE
while read url; do
echo "Processing $url..."
FILENAME=$(basename "$url")
curl -s "$url" -o "/tmp/$FILENAME"
pdftotext "/tmp/$FILENAME" "/tmp/${FILENAME}.txt"
echo "--- $FILENAME ---" >> $OUTPUT_FILE
grep -i -E "per host|per GB|per month|billing" "/tmp/${FILENAME}.txt" | head -5 >> $OUTPUT_FILE
echo "" >> $OUTPUT_FILE
done < "$PDF_LIST"
echo "Report saved to $OUTPUT_FILE"
```
Has anyone else tried something like this? I'm sure there are better tools (maybe even something in Prometheus/Grafana for tracking changes?), but I was happy to get this running 😅. Would love to hear how you handle doc comparison.
So you're scraping their pricing pages with `curl`. Did you check the robots.txt on any of those vendor sites first? Or consider what happens when they change their PDF structure next month? That grep for "per host" is going to miss any regional pricing tables or private offers.
Also, you're leaving PDFs in /tmp for anyone on that box to read. Not a huge deal for public data, but you should at least `curl -s "$url" | pdftotext - -` and pipe it instead.
You're tracking costs, which is good. But you're missing the real nightmare: commitment discounts, egress fees, and support add-ons. Those are usually buried in separate docs.
- Nina
That's really clever! I was just looking at Datadog's pricing the other day and it was such a headache. I never thought about trying to automate it like that.
When you run your grep for things like "per host," are you catching all the different plan names too? I got lost between their "Pro" and "Enterprise" columns.
Also, do you have this running somewhere regularly, or just when you need a fresh snapshot?
That's a really interesting approach. I've been looking at vendor pricing PDFs too, and I've found they sometimes list the same thing in multiple sections with slightly different wording. Your grep for "per host" might miss something like "monthly cost per monitored host."
Have you thought about adding a step to look for common units like "GB," "TB," or "hour"?
Good point about the wording variations. That's why regex alone usually fails after the first minor PDF update.
Adding unit searches helps, but you'll still miss the real cost drivers. You need to parse the actual pricing tables, not just lines with "GB." Vendors love burying minimum monthly fees in the footnotes of those tables.
And none of this catches the annual commit discounts, which often aren't even in the main pricing PDF. You're just automating the wrong 20% of the problem.
show me the bill
You're absolutely right about the annual commits. I've had to call sales reps directly for those discounted rate cards more times than I can count. They're intentionally kept off the public PDFs.
The footnote point is a killer too. A table might show $0.10 per GB, but the tiny print says "minimum $500 monthly per product." You can grep all day and still miss that.
It's a good first script for the baseline, but you're automating a moving target. The real number is often a conversation.
Keep it civil, keep it real.