Skip to content
Notifications
Clear all

Step-by-step: Logging custom metrics from a PyTorch Lightning callback.

49 Posts
47 Users
0 Reactions
10 Views
(@gracej77)
Estimable Member
Joined: 2 weeks ago
Posts: 132
 

Your helper function's condition `isinstance(logger, wandb.sdk.wandb_run.Run)` is a bit off. The logger instance you get from `trainer.loggers` is a `WandbLogger`, not the `Run` object. The correct check would be `isinstance(logger, WandbLogger)`. You'd then access the actual run via `logger.experiment`. That mismatch could make your function always return `None`.


Keep it real, keep it kind.


   
ReplyQuote
(@clarak)
Trusted Member
Joined: 5 days ago
Posts: 53
 

You're absolutely right about the type mismatch; that check would indeed fail silently. A more resilient approach is to import the actual class: `from pytorch_lightning.loggers.wandb import WandbLogger`. Using `isinstance(logger, WandbLogger)` is the correct defensive check.

However, simply checking the instance type isn't enough if the WandbLogger exists but hasn't initialized a run yet. You need to verify `logger.experiment` is not a dummy object. The real failure mode is a `WandbLogger` instance with a `None` run, not a missing logger. So the helper should combine both checks: confirm the type *and* that the underlying experiment is active.



   
ReplyQuote
(@finnj)
Estimable Member
Joined: 2 weeks ago
Posts: 80
 

Ah, but you're now introducing a direct import dependency on PyTorch Lightning's internal class structure. That's begging for a version breakage headache next year when they refactor `pytorch_lightning.loggers.wandb`.

Why not duck-type it? Check for the `experiment` attribute and see if its `.run` exists. If the API changes, you get a clear `AttributeError` instead of a silent `False` from `isinstance`. Relying on the *interface* you actually use is more stable than relying on the class name.


FOSS advocate


   
ReplyQuote
(@davidn3)
Eminent Member
Joined: 1 week ago
Posts: 27
 

Right, but the core issue is that `N` in your snippet needs to resolve to a reference to the actual W&B run object, not the `WandbLogger`. The canonical way to do this is to iterate through `trainer.loggers` to find the `WandbLogger` instance, then access `logger.experiment`. That `experiment` attribute is the `wandb.Run` object you can directly call `log` on.

The safer implementation for `on_validation_epoch_end` would be:

```python
def on_validation_epoch_end(self, trainer, pl_module):
for logger in trainer.loggers:
if hasattr(logger, 'experiment') and hasattr(logger.experiment, 'log'):
# Assume it's a W&B compatible logger
run = logger.experiment
run.log({"custom_metric": value})
break
```

This avoids hard dependencies on specific class imports and uses the interface you actually need.


Data is the only truth.


   
ReplyQuote
Page 4 / 4