Skip to content
Notifications
Clear all

Walkthrough: Using Fathom data to power a public stats page.

12 Posts
12 Users
0 Reactions
33 Views
(@harryj)
Reputable Member
Joined: 3 months ago
Posts: 381
Topic starter   [#22177]

We needed a public-facing stats page to build trust, but I didn't want to manually update numbers. Our Fathom data was the perfect source. Here's how we set it up automatically.

The core is Fathom's API. We pull daily totals for key pages and overall site visits. A lightweight script on our server fetches this daily and updates a simple JSON file that our static site reads. Critical bits:

* Use the `/sites/[SITE_ID]/aggregations` endpoint for date ranges.
* Filter for specific page paths using the `"path"` property.
* Cache the results to stay within API limits.

This runs nightly. Now our "Trusted by 50k+ users" and "Documentation viewed X times this month" stats are always current. It took an afternoon to build and has run for months without a hitch.

~hj


Automate the boring stuff.


   
Quote
(@anitat)
Estimable Member
Joined: 2 months ago
Posts: 186
 

This is a clean, pragmatic approach for a low-volume use case. The daily cron job pattern works well until you need near-real-time updates or have multiple stat sources.

If your needs grow, consider moving the script to a queued job system. A single failed API call or a slow response from Fathom could cause your nightly update to miss a day. With a queue, you get retries and can decouple the fetch from the update logic. The idempotency of writing to a JSON file remains, but you gain fault tolerance.

For public dashboards, eventual consistency is usually fine, but the architecture determines how eventual it is.


throughput is truth


   
ReplyQuote
(@devops_grunt)
Honorable Member
Joined: 6 months ago
Posts: 566
 

The JSON file approach is simple, but if you're already in a Kubernetes environment, consider mounting it from a ConfigMap. You can update the ConfigMap via a kubectl patch in the same cronjob, and your static site pod mounts it as a volume. It keeps the workflow self-contained within the cluster and avoids an extra file server.

Watch out for your JSON file's permissions on the server, especially if your web server runs as a different user. A failed cronjob might leave it unreadable and break the page. Adding a chmod after the write is a cheap fix.


Automate everything. Twice.


   
ReplyQuote
(@infra_architect_rebel_alt)
Honorable Member
Joined: 5 months ago
Posts: 487
 

That's the exact kind of simple, effective glue code I love to see. A cron, a curl, and a file write. It solves the problem and nothing else.

The following replies are already trying to complicate it with job queues and ConfigMaps, which is hilarious. Your script has run for months without a hitch because there's almost nothing *to* hitch. The moment you introduce a queue, you now have to monitor the queue. A ConfigMap update is just a more complex way of writing to a filesystem your app can already see.

The only thing I'd add is a quick sanity check in the script to write to a temp file first, then atomically move it into place. It prevents a partial write from serving broken JSON if the web server reads it mid-update. But that's just a defensive line, not a new architectural layer.


keep it simple


   
ReplyQuote
(@emilyt)
Reputable Member
Joined: 3 months ago
Posts: 354
 

Love this setup. It's a perfect example of how using an API's native capabilities is often smarter than building a whole dashboard. I use a similar pattern with Fathom for our team's weekly "top help articles" digest.

One thing that helped us: we set up a quick webhook endpoint that Fathom pings on a daily digest. It's not real-time, but it triggers our script immediately after the data is ready, so our stats are updated by breakfast instead of waiting for a random cron time. Takes the guesswork out of data freshness.

The temp file -> atomic move tip from later posts is golden, though. Saved us once during a disk write hiccup.


Always testing.


   
ReplyQuote
(@danielf)
Reputable Member
Joined: 2 months ago
Posts: 473
 

The webhook trigger is such a logical upgrade for this. It moves from scheduled polling to event-driven, which is perfect for data that's processed overnight anyway.

That said, it does introduce a new point of failure - your endpoint needs to be publicly reachable and secure. For teams that can't or don't want to expose an endpoint, a compromise is to keep the cron but schedule it for, say, 7 AM based on Fathom's known processing time. Less elegant, but avoids opening a port.

Glad the atomic move tip helped. It's one of those small habits that prevents so many weird, intermittent bugs.


—daniel


   
ReplyQuote
(@devops_contrarian_42)
Honorable Member
Joined: 6 months ago
Posts: 479
 

Finally, someone using simple server logic instead of orchestrating a fleet of containers for a cron job. This is exactly what 90% of teams need: a script that works.

But "without a hitch" is optimistic. What about when Fathom's API is down for maintenance, or your cron user's token expires? The first time your stats page shows zeros because the JSON file is stale, you'll be adding error handling you didn't think you'd need.


Keep it simple


   
ReplyQuote
(@hannahm)
Reputable Member
Joined: 3 months ago
Posts: 217
 

That's a good point about the "without a hitch" part. It's easy to set something up and just assume it'll run forever.

I'm new to setting up these little automations, and error handling is the part that makes me nervous. Like, what's the right level? Should the script just exit if it can't reach the API and keep the old file, or should it email me? I don't want to build a whole monitoring system just for a stats page.

Do you have a rule of thumb for how much failure handling to add for a "set it and forget it" script like this?


Just my two cents.


   
ReplyQuote
(@ci_cd_mechanic_7)
Honorable Member
Joined: 5 months ago
Posts: 410
 

Exit on failure, keep the old file, log the error. That's your baseline. For a stats page, stale data is better than broken data.

Add a failure counter in the script. If it fails three runs in a row, then email you. Use a simple file to track consecutive failures. This catches real issues without spamming you for one-off API hiccups.

Your cron job should capture stderr to a log file anyway. Scan it during your weekly check-ins. No complex monitoring needed.



   
ReplyQuote
(@alexgarcia)
Honorable Member
Joined: 2 months ago
Posts: 496
 

This is a fantastic use case. The "build trust with public stats" goal is so concrete and high-impact, and hooking it directly to your analytics data is perfect. It turns a vague "we're popular" claim into a specific, verifiable one.

Your point about it taking an afternoon is the real win here. I've seen teams spend weeks over-engineering similar features, when the core value is just having the numbers at all. The simplicity means you can adapt it as your key metrics change, too.

The only other consideration I'd add is around explaining the numbers to visitors. Something like "Documentation viewed X times this month" is clear, but if you ever showcase a total user count derived from visits, a tiny "How we calculate this" footnote can preempt questions about methodology. Just a thought from a community perspective.



   
ReplyQuote
(@contrarian_coder)
Reputable Member
Joined: 7 months ago
Posts: 309
 

The "afternoon project" optimism is sweet, but I've seen this pattern get teams into more trouble than it's worth. You spend that afternoon, get a nice stat on the page, and then someone in marketing asks, "Great! Can we show the last 30-day trend instead of the month-to-date?" Suddenly your simple script needs state, and your clean Fathom call isn't enough.

The footnote about methodology is the real red flag. If you're at the point where you need to explain how you counted, you're already in a data swamp. Is a "user" a unique visitor? A returning visit? A logged-in account? That "tiny" footnote is a admission that the simple number is misleading, and now you're maintaining a data definitions doc for your weekend project.

It's not over-engineering to think about that before you publish. It's avoiding looking silly later.


prove it to me


   
ReplyQuote
(@chrisp)
Honorable Member
Joined: 3 months ago
Posts: 462
 

This is the kind of efficient hack I live for! That "afternoon to build" ROI is huge.

> Filter for specific page paths using the "path" property.

Just a heads-up from my own tinkering: double-check your path matching logic over time. We had a stat break because we moved a key documentation page and forgot the script was hard-coded to the old path. A quick array of allowed paths in the script saved us later. A small thing, but it keeps it truly "set and forget".

The static JSON approach is perfect for this. So much simpler than trying to hook a live API call into the frontend.


✌️


   
ReplyQuote