Skip to content

No parameter update with RTE #9

@nguyen-an13

Description

@nguyen-an13

Dear authors,

Thank you for your contribution.
I would like to share that in the case of rte.sh, it seems that there is no update, the accuracy is kept around 52% for a long time.
Could you please have a look at nlu/HRA.
Thank you!

RTE
12/16/2025 13:49:32|INFO|RTE|00| None[96.4%][-0.01h] Steps=1500, loss=0.6831837543646495, examples=47886, loss_scale=16384.0, 6.3s
Evaluating: 001500-1556: 100%|████████████████████| 2/2 [00:00<00:00, 3.12it/s]
12/16/2025 13:49:33|INFO|RTE|00| ***** Eval results-dev-001500-1556 *****
12/16/2025 13:49:33|INFO|RTE|00| accuracy = 0.5270758122743683
12/16/2025 13:49:33|INFO|RTE|00| eval_loss = 0.6906195282936096
12/16/2025 13:49:33|INFO|RTE|00| eval_metric = 0.5270758122743683
12/16/2025 13:49:33|INFO|RTE|00| eval_samples = 277
12/16/2025 13:49:33|INFO|RTE|00| Best metric: 0.5270758122743683@78
12/16/2025 13:49:48|INFO|RTE|00| None[100.0%][-0.00h] Steps=1556, loss=0.6811035534984655, examples=49678, loss_scale=16384.0, 16.4s
Evaluating: 001556-1556: 100%|████████████████████| 2/2 [00:00<00:00, 3.63it/s]
12/16/2025 13:49:49|INFO|RTE|00| ***** Eval results-dev-001556-1556 *****
12/16/2025 13:49:49|INFO|RTE|00| accuracy = 0.5270758122743683
12/16/2025 13:49:49|INFO|RTE|00| eval_loss = 0.69061279296875
12/16/2025 13:49:49|INFO|RTE|00| eval_metric = 0.5270758122743683
12/16/2025 13:49:49|INFO|RTE|00| eval_samples = 277
12/16/2025 13:49:49|INFO|RTE|00| Best metric: 0.5270758122743683@78
Evaluating: deberta-v3-base: 100%|██████████████| 12/12 [00:03<00:00, 3.23it/s]
12/16/2025 13:49:53|INFO|RTE|00| ***** Dump prediction results-test-deberta-v3-base *****
12/16/2025 13:49:53|INFO|RTE|00| Location: /tmp/DeBERTa//outputs/deberta-v3-base/RTE/test_logits_test_deberta-v3-base.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions