zero-shot

[jeswan]

Issue by pasinit
Wednesday Apr 15, 2020 at 22:17 GMT
Originally opened as nyu-mll/jiant#1068

---

Dear all,
I was wondering if there is a way to train a model on a task and then test on multiple target task without doing any additional training on each of them nor having dedicated parameters in the models for those tasks. IN other words, is it possible to have a zero-shot setting where one train on a dataset in one language and test in other languages without performing training on each of them?

Thanks

[jeswan]

Comment by pruksmhc
Wednesday Apr 15, 2020 at 22:21 GMT

---

There is a way to accomplish this in jiant, but it is not very intuitive - that is currently what we do with the diagnostic tasks in this repo. Here's a thread about it.
nyu-mll/jiant#1056
In terms of testing in other languages, right now, we only support English-language tasks. You would have to add tasks with datasets in other tasks. Please refer to our tutorial on adding tasks: https://github.com/nyu-mll/jiant/blob/master/tutorials/adding_tasks.md

[jeswan]

Comment by pasinit
Wednesday Apr 15, 2020 at 22:27 GMT

---

thanks :)