fix(engine): allocate distributed init ports outside the ephemeral range

[CAICAIIs]

What does this PR do?

Fixes #517: distributed worker startup could fail with EADDRINUSE on busy
hosts. find_free_port() probed a port with bind(0) and immediately closed
the reservation socket; the just-freed ephemeral port could be re-taken as
the source port of an outbound connection (leaving it in TIME-WAIT, so the
later TCPStore bind failed), and consecutive calls could hand the same port
to the train and rollout clusters.

New approach (per maintainer review): the coordinator creates and retains
the rendezvous store. Instead of probing a port, the coordinator now creates
a TCPStore(port=0) and keeps it open for the cluster lifetime, so the
resolved port is genuinely reserved before any worker starts. Workers join as
TCPStore clients and initialize the process group with store=, so no
worker ever binds the rendezvous port — the bind-close-bind TOCTOU race is
eliminated, including across processes.

Changes:

• areno/engine/protocol.py: new _create_rendezvous_store() binds port=0
  and is retained on the cluster; Cluster.start and
  start_partitioned_clusters resolve the store port into the shared
  world_spec before spawning workers. find_free_port is removed.
• areno/engine/parallel/context.py: init_process_group creates a client
  TCPStore and passes it to dist.init_process_group(store=...).
• areno/api/backend/cuda/backend.py: the rollout world_spec uses a
  placeholder port that start_partitioned_clusters fills in.
• Tests: the gloo broadcast test now mirrors production with a coordinator
  store; a new test asserts the resolved port is genuinely held (re-binding
  the same port fails while the store is alive) and serves client stores.

No public API or CLI change.

Repro evidence

On a shared 8×A100 node, 4 consecutive areno train startups failed with
DistNetworkError: ... EADDRINUSE on freshly "free" ports (39625 / 34127 /
58051 / 49989); ss at failure time showed the chosen port in TIME-WAIT as
the source of an outbound connection. With the coordinator-held store,
startups on the same busy host succeed reliably.

Test commands (CPU)

pytest tests/test_parallel_partition_cpu.py -v

The gloo broadcast test spawns 4 worker processes that join through the
coordinator-held store (client stores + init_process_group(store=...)),
exercising the production rendezvous path end to end.

Type of change

• 🐛 Bug fix
• ✨ New feature
• 💥 Breaking change
• 📝 Documentation update
• ♻️ Refactoring
• ⚡ Performance improvement
• ✅ Test coverage improvement

Checklist

• The PR title summarizes the contribution.
• Linked the related issue in the description (if any).
• Existing tests pass (pytest tests/ -k cpu).
• New behavior is covered by tests.
• Described the test commands run and any hardware limitations.
• Public API / CLI changes are additive and backward-compatible (see CONTRIBUTING.md).

[xsuler]

Hi @CAICAIIs , your change mitigates the observed collision but does not eliminate the bind-close-bind TOCTOU race, especially across processes. A stronger approach is to let the coordinator create and retain a TCPStore with port=0, then pass its resolved port to worker-side client stores. Also, the exclusion check/add should currently be under the same lock, and the fallback reintroduces the original ephemeral-port issue.

[CAICAIIs]

Thanks @xsuler — reworked per your suggestion: the coordinator now creates and retains the TCPStore(port=0) and workers join as client stores (find_free_port removed, fallback gone); the PR is rebased onto the latest main and the conflict is resolved.