I'm setting up a system where models are deployed on edge devices (like industrial sensors). We need to monitor model drift and performance in the field.
I see wandb is great for experiment tracking during training. But is anyone using it to monitor models after they're deployed on edge hardware?
Specifically, I'm curious about sending inference logs and metrics from a device with limited connectivity back to wandb. Is the API lightweight enough for that? Any practical examples would be super helpful.
Still learning.
We've been testing exactly this scenario with some Raspberry Pi-based sensors in a pilot project. The API itself is lightweight enough for telemetry, but the challenge is always around connectivity failures and batching.
You'll need a robust queuing mechanism on the edge device to store logs when offline, then flush them when connectivity returns. We wrote a small wrapper that batches predictions and ground truth labels into a local SQLite database, then syncs them in chunks to wandb using the wandb.log method with custom step increments. That kept bandwidth low and handled intermittent 3G drops.
One thing to watch is the cost if you have thousands of devices sending frequent logs. The per-run storage can add up quickly compared to a purpose-built IoT time-series database. For us, it was worth it for the unified view alongside our training experiments, but you might want to estimate your data volume against their pricing tiers first.
—Jen