[Reorgnize] Reorganize intra-node examples add add tests

examples/distributed/README.md

@@ -1,13 +1,11 @@
 # Distributed Examples
 
 This directory contains examples demonstrating distributed computing capabilities using TileLang.
+These examples are sorted into two categories:
+- Examples under `nvshmem` folder and inter-node examples depend on NVSHMEM library for distributed communication.
+- Other examples have no external dependency and only rely on TileScale IPC
 
-For example,
-```
-./tilelang/distributed/launch.sh examples/distributed/example_allgather.py
-```
-
-## Prerequisites
+## `nvshmem` examples
 
 Before running the examples, you need to build NVSHMEM library for device-side code generation.
 
@@ -28,3 +26,15 @@ Then you can test python import:
 ```bash
 python -c "import pynvshmem"
 ```
+
+Finally, run examples like this:
+```bash
+TILELANG_USE_DISTRIBUTED=1 bash ./tilelang/distributed/launch.sh examples/distributed/nvshmem/example_allgather.py
+```
+
+## IPC-based examples
+
+Simply run via python:
+```bash
+TILELANG_USE_DISTRIBUTED=1 python examples/distributed/intranode/example_allgather_gemm_overlapped.py
+```

examples/distributed/example_gemm_rs_overlapped.py → examples/distributed/intranode/example_gemm_rs_overlapped.py

@@ -1,5 +1,3 @@
-from __future__ import annotations
-
 import tilelang
 import tilelang.language as T
 import argparse