Huber Loss and Log Cosh Loss

[dnth]

Has anyone tried the Huber Loss and Log Cosh Loss on time series dataset?
Below is something I tried and it seems to work with my dataset. Not sure if this is the right way of doing custom loss functions.

class LogCoshLoss(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, y_t, y_prime_t):
        ey_t = y_t - y_prime_t
        return torch.mean(torch.log(torch.cosh(ey_t + 1e-12)))
    

def HuberFlat(*args, axis=-1, floatify=True, **kwargs):
    "Same as `nn.SmoothL1Loss`, but flattens input and target."
    return BaseLoss(nn.SmoothL1Loss, *args, axis=axis, floatify=floatify, is_2d=False, **kwargs)

def LogCoshFlat(*args, axis=-1, floatify=True, **kwargs):
    "Same as `nn.SmoothL1Loss`, but flattens input and target."
    return BaseLoss(LogCoshLoss, *args, axis=axis, floatify=floatify, is_2d=False, **kwargs)


learn = Learner(dls, model, metrics=[rmse, mse], opt_func=Adam, loss_func=LogCoshFlat())

[oguiza]

Hi,
I haven't personally used those losses you mention.
As to how to build a custom loss, there are many examples in fastai losses. It's usually a good idea to test a few values for which you know what the output should be and confirm it's the same you are actually getting.

[dnth]

Hi @oguiza , I notice your commit on Huber Loss and Log Cosh Loss thank you for adding these losses in TSAI! Have you gotten good results from those losses?

[oguiza]

I don't have much experience using them. But in my case, I ended up using nn.L1Loss. But I guess they may be useful in other cases.

[oguiza]

BTW, when I added the 2 loss functions, I hadn't seen you discussion post. For some reason, GitHub doesn't alert me of new discussions even if I'm watching all activity :(
Otherwise, I would have added your implementations. Sorry about that!

[dnth]

Oh! Don't worry about it. I have ZERO issues with that! I am grateful that TSAI now supports the loss functions :)