transformers version

[wangjian026]

hello，i use the transformers(v3.0.2),and got multiple error "unresolved reference" in modeling_bart.py. if i use higher version of transformers(v3.1.0), the above error disappears，However, at running time，since there are great differences between the two versions， there is something wrong occur in function "prepare_inputs_for_generation()" in modeling_bart.py.

[elevenofji]

not only the "prepare_inputs_for_generation", in lighting_base.py, "Adafactor" and "get_polynomial_decay_schedule_with_warmup" cannot be imported correctly.I'm just trying to find a solution

[wangjian026]

use transformers(v3.2.0), and change the code"encoder_outputs, decoder_past_key_values = past" of function"prepare_inputs_for_generation" in lighting_base.py to "encoder_outputs, decoder_past_key_values = (kwargs['encoder_outputs'],None)" may solve this problem.

[elevenofji]

use transformers(v3.2.0), and change the code"encoder_outputs, decoder_past_key_values = past" of function"prepare_inputs_for_generation" in lighting_base.py to "encoder_outputs, decoder_past_key_values = (kwargs['encoder_outputs'],None)" may solve this problem.

GOOD idea. And I think sth about decoder_past_key_values, such as "decoder_past_key_values = outputs.decoder_past_key_values," in modeling_bart should be removed too.

BTW, have you tried the model with answer_relevance_score? I just found that it depends on another paper and there's a ckpt too. (https://github.com/HLTCHKUST/HLTC-MRQA)

[Iamfinethanksu]

Thank you for the discussion. Actually, our transformer's version for the experiment is a little bit twisted between the two version, for some historical issue. You may choose v3.0.2 and modify some function (input, or output ) accordingly, then it should be fine.

[elevenofji]

@Iamfinethanksu Yeah，u r right. The transformer's version has to be 3.0.2, and there're so many differences in the "input" and "output" between 3.0.2 and 3.2.0. I cannot get an expected result in 3.2.0 (even cannot get a normal decoding sentence, the reason may be that @wangjian026 solution may ignore the "decoder_past_key_values" directly. ). After fixing the transformer's version to 3.0.2 and changing some funcs, it works. I'll push the changed code to my repos.

Here are 10 hours fine-tuning results (without mrc's score).

Here are paper's res:

The reason may be that the model takes "rouge2" as a val-metric.

@Iamfinethanksu Maybe you can emphasize the problem in README.md. (Also, the paper pdf link in README.md is wrong, linking to your another paper in emnlp19 XD).
Thx for @Iamfinethanksu and @wangjian026 's help.

[wangjian026]

@Iamfinethanksu Yeah，u r right. The transformer's version has to be 3.0.2, and there're so many differences in the "input" and "output" between 3.0.2 and 3.2.0. I cannot get an expected result in 3.2.0 (even cannot get a normal decoding sentence, the reason may be that @wangjian026 solution may ignore the "decoder_past_key_values" directly. ). After fixing the transformer's version to 3.0.2 and changing some funcs, it works. I'll push the changed code to my repos.

Here are 10 hours fine-tuning results (without mrc's score).

Here are paper's res:

The reason may be that the model takes "rouge2" as a val-metric.

@Iamfinethanksu Maybe you can emphasize the problem in README.md. (Also, the paper pdf link in README.md is wrong, linking to your another paper in emnlp19 XD). Thx for @Iamfinethanksu and @wangjian026 's help.

GOOD idea！I have been trying to get a better result, but I failed. Could you send me the modified codes with transformer's version 3.0.2 ? Thanks again.Thanks again, my email: [email protected]

[elevenofji]

@Iamfinethanksu Yeah，u r right. The transformer's version has to be 3.0.2, and there're so many differences in the "input" and "output" between 3.0.2 and 3.2.0. I cannot get an expected result in 3.2.0 (even cannot get a normal decoding sentence, the reason may be that @wangjian026 solution may ignore the "decoder_past_key_values" directly. ). After fixing the transformer's version to 3.0.2 and changing some funcs, it works. I'll push the changed code to my repos.
Here are 10 hours fine-tuning results (without mrc's score).
Here are paper's res:
The reason may be that the model takes "rouge2" as a val-metric.
@Iamfinethanksu Maybe you can emphasize the problem in README.md. (Also, the paper pdf link in README.md is wrong, linking to your another paper in emnlp19 XD). Thx for @Iamfinethanksu and @wangjian026 's help.

GOOD idea！I have been trying to get a better result, but I failed. Could you send me the modified codes with transformer's version 3.0.2 ? Thanks again.Thanks again, my email: [email protected]

you can just download in https://github.com/elevenofji/bart_test.

[wangjian026]

@Iamfinethanksu Yeah，u r right. The transformer's version has to be 3.0.2, and there're so many differences in the "input" and "output" between 3.0.2 and 3.2.0. I cannot get an expected result in 3.2.0 (even cannot get a normal decoding sentence, the reason may be that @wangjian026 solution may ignore the "decoder_past_key_values" directly. ). After fixing the transformer's version to 3.0.2 and changing some funcs, it works. I'll push the changed code to my repos.
Here are 10 hours fine-tuning results (without mrc's score).
Here are paper's res:
The reason may be that the model takes "rouge2" as a val-metric.
@Iamfinethanksu Maybe you can emphasize the problem in README.md. (Also, the paper pdf link in README.md is wrong, linking to your another paper in emnlp19 XD). Thx for @Iamfinethanksu and @wangjian026 's help.

GOOD idea！I have been trying to get a better result, but I failed. Could you send me the modified codes with transformer's version 3.0.2 ? Thanks again.Thanks again, my email: [email protected]

you can just download in https://github.com/elevenofji/bart_test.

Thank you very much for your work, I have downloaded the code, however, a file named modeling_bart seems to have been lost in the src file

[elevenofji]

@Iamfinethanksu Yeah，u r right. The transformer's version has to be 3.0.2, and there're so many differences in the "input" and "output" between 3.0.2 and 3.2.0. I cannot get an expected result in 3.2.0 (even cannot get a normal decoding sentence, the reason may be that @wangjian026 solution may ignore the "decoder_past_key_values" directly. ). After fixing the transformer's version to 3.0.2 and changing some funcs, it works. I'll push the changed code to my repos.
Here are 10 hours fine-tuning results (without mrc's score).
Here are paper's res:
The reason may be that the model takes "rouge2" as a val-metric.
@Iamfinethanksu Maybe you can emphasize the problem in README.md. (Also, the paper pdf link in README.md is wrong, linking to your another paper in emnlp19 XD). Thx for @Iamfinethanksu and @wangjian026 's help.

GOOD idea！I have been trying to get a better result, but I failed. Could you send me the modified codes with transformer's version 3.0.2 ? Thanks again.Thanks again, my email: [email protected]

you can just download in https://github.com/elevenofji/bart_test.

Thank you very much for your work, I have downloaded the code, however, a file named modeling_bart seems to have been lost in the src file

Sorry for that. I've fixed. Maybe we can discuss in https://github.com/elevenofji/bart_test. This issue can be closed.