Accuracy function

[PonteIneptique]

Actually duplicates in part #53 because I forgot to go back to master...

This adds the ability to chose on which metric to fit in regard to devset: accuracy, f1, precision, recall, balanced-accuracy.

Default value should not change pas behaviours.

[PonteIneptique]

The LR Scheduler does not seem to update with this PR. I need to get into the details here but I got 25 steps for lemmatization with lr_patience 10 and patience 8 which definitely does not seem right.

[emanjavacas]

Did you find out why?

[PonteIneptique]

I'll go back into having a look at this.

[PonteIneptique]

This is good to go.

[PonteIneptique]

I was a bit surprised (also) by the fact we still print threshold value of LROnPlateau but we do not set it.

pie/pie/trainer.py

Lines 202 to 204 in 230e0e6

	self.lr_scheduler = LRScheduler(
	self.optimizer, factor=settings.lr_factor,
	patience=settings.lr_patience, min_lr=settings.min_lr)

pie/pie/trainer.py

Lines 154 to 155 in 230e0e6

	self.lr_scheduler = optim.lr_scheduler.ReduceLROnPlateau(
	optimizer, mode='max', threshold=threshold, **kwargs)

It feels to me we should forward the value of the target task but :)

[emanjavacas]

What was the issue?

Re: threshold. You are right. I never got to use it and was kept a 0.0. However, note that there are 2 thresholds here. One is for the LR schedule, the other is for the task schedule. Just saying because it's not very well documented...

[PonteIneptique]

I don't know, probably had a misunderstanding or did not check correctly. There is basically no reason it should affect lr schedule... As for the second comment, yes, maybe we should have a lr_threshold? Le mar. 4 août 2020 à 5:10 PM, Enrique Manjavacas <[email protected]> a écrit :

…

What was the issue? Re: threshold. You are right. I never got to use it and was kept a 0.0. However, note that there are 2 thresholds here. One is for the LR schedule, the other is for the task schedule. Just saying because it's not very well documented... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#54 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOXEZXJWKZQUM6OJHAZBBDR7AQHZANCNFSM4LEMOL6Q> .

[PonteIneptique]

Note: I think if you merge this Optuna will need to be changed a litte :)