[Tests] Reintroduce VMAS football

[matteobettini]

Vmas fooball bug should be fixed

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3178

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 17 New Failures, 2 Cancelled Jobs, 3 Unrelated Failures

As of commit da75c57 with merge base 8374bb4 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t da2341478b55bdb44a8795bfb4776c7d43c488fa505dc5a6b1c24c184c24abb7 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t a35c097b20e62e9334bf108aad1c0fb66e0b536d55b83b27223ddc23ae7d3bef /exec failed with exit code 1
• Libs Tests on Linux / unittests-envpool (3.10, 12.8) / linux-job (gh)
  test/test_libs.py::TestD4RL::test_d4rl_iteration[False-walker2d-medium-replay-v2]
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 9a5d6b0249d7936d47b04f6f1623fe4010f151b702414d73cc6546eedcd7803a /exec failed with exit code 1
• Libs Tests on Linux / unittests-jumanji (3.10, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t cb5c64a071e48af43f93acc402d6b794186267ed30ebdd21d677d64ea5b8d0bc /exec failed with exit code 250
• Libs Tests on Linux / unittests-vmas (3.10, 12.8) / linux-job (gh)
  test/test_libs.py::TestVmas::test_vmas_seeding[discovery]
• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t 5371e99fa9bb7131495bc92dc0417c613cf316827f027f284341aad6764da135 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t 2852ac0147cabb48deb492712a8b81e70b81d26b6159a305fdebda3b31e0a5c8 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  RuntimeError: Command docker exec -t 050eed3da70e0c0624ab3f06be0e706e51c5b2a69e50fd3433b89ad7c68ee654 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  RuntimeError: SumSegmentTreeFp32 is not available. See warning above.
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_env.py::TestNonTensorEnv::test_parallel[False-False]
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 8b3049dfc0c99306cd9a1f5c0bdea8a8af81456f1ee4dfbac97a60576963df82 /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 695de6bc689c63cbe5f7bd55d7184a62829d545b903c9ec4980927502c847f42 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh)
  test/test_trainer.py::TestRB::test_rb_trainer_save[True-torch-tensor-True]
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh)
  test/test_trainer.py::TestRB::test_rb_trainer_save[True-torch-tensor-True]

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  ##[error]The operation was canceled.
• Continuous Benchmark / GPU Pytest benchmark (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Libs Tests on Linux / unittests-pettingzoo / linux-job (gh) (trunk failure)
  test/test_libs.py::TestPettingZoo::test_dead_agents_done
• Libs Tests on Linux / unittests-robohive (3.9, 12.8) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• LLM Tests on Linux / unittests (3.9, 12.8) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[matteobettini]

@vmoens can you activate the vmas CI for the PR pls?

[vmoens]

LGTM
Waiting for the tests to complete this time

[matteobettini]

We may need to add the label and rerun no?

[vmoens]

Merging as football is fixed but we have a new error

FAILED test/test_libs.py::TestVmas::test_vmas_seeding[simple_reference] - AssertionError: key ('agents', 'action') does not match, got mse = 0.4989
FAILED test/test_libs.py::TestVmas::test_vmas_seeding[waterfall] - AssertionError: key ('agents', 'action') does not match, got mse = 0.8571
FAILED test/test_libs.py::TestVmas::test_vmas_seeding[flocking] - AssertionError: key ('agents', 'action') does not match, got mse = 0.8473
FAILED test/test_libs.py::TestVmas::test_vmas_seeding[discovery] - AssertionError: key ('agents', 'action') does not match, got mse = 0.8666

[matteobettini]

Thanks!

Ah the seeding. This comes from our super cool code for setting a local torch seed, lemme see if the seeding tests still male sense

[vmoens]

Thanks
As always, you're blazing fast, we'll call you Claude from now on