batch vs epoch calculation for custom metric

[sachinruk]

Suppose I have a metric that looks like so (straight from docs):

class MyAccuracy(Metric):
    def __init__(self, dist_sync_on_step=False):
        super().__init__(dist_sync_on_step=dist_sync_on_step)

        self.add_state("correct", default=torch.tensor(0), dist_reduce_fx="sum")
        self.add_state("total", default=torch.tensor(0), dist_reduce_fx="sum")

    def update(self, preds: torch.Tensor, target: torch.Tensor):
        preds, target = self._input_format(preds, target)
        assert preds.shape == target.shape

        self.correct += torch.sum(preds == target)
        self.total += target.numel()

    def compute(self):
        return self.correct.float() / self.total

If I declare this inside my LightningModule init as self.valid_acc = MyAccuracy() and inside validation step if I do self.log('valid_acc', self.valid_acc, on_step=True, on_epoch=True) how would it know to differentiate between the batch metrics vs epoch metrics. To my understanding it's just calling compute in each case where the batch accuracy is overridden by self.correct.float() / self.total where collect and total are values accumulated so far.

Otherwise do I need to declare two metrics in my init for the batch and epoch one separately?

[rahulvigneswaran]

Hey @sachinruk, If I remember it correctly, self.correct and self.total are cached. So when you do self.log('valid_acc', self.valid_acc, on_step=True, on_epoch=True), it will both do a self.compute at batch level and epoch level using the cached values.

The logger will have a corresponding "step" and "epoch" suffix. So you don't have to declare two different metrics.