[Feature Request] Pluggable Metrics&HG integration

[burakisikli]

Hi,
It would be nice to have pluggable metrics that everyone can use without explicitly use the model code. Also, it would be nice to have huggingface transformer library integration so that every transformer model can be used. I can help as much as I can

Thanks
Best

[znado]

Hey!

For metrics we typically depend on Robustness Metrics although there maybe a few places we've rolled our own.

As for huggingface, for example do you mean being able to add a different type of model besides BERT here? It would be very interesting to see what this would do, and we'll gladly accept pull requests for huggingface_* version(s) of baselines.