Skip to content

Commit 0207190

Browse files
author
hongtaozhang
committed
1. add docs; 2. add example; 3. fix comments.
1 parent fd87b33 commit 0207190

File tree

3 files changed

+112
-1
lines changed

3 files changed

+112
-1
lines changed

‎docs/user-tutorial/benchmarks/micro-benchmarks.md

+75
Original file line numberDiff line numberDiff line change
@@ -384,6 +384,81 @@ with topology distance of 2, 4, 6, respectively.
384384
| ib-traffic/ib\_write\_bw\_${msg_size}\_${direction}\_${line}\_${pair}:${server}\_${client} | bandwidth (GB/s) | The max bandwidth of perftest (ib_write_bw, ib_send_bw, ib_read_bw) using ${msg_size} with ${direction}('cpu-to-cpu'/'gpu-to-gpu'/'gpu-to-cpu'/'cpu-to-gpu') run between the ${pair}<sup>th</sup> node pair in the ${line}<sup>th</sup> line of the config, ${server} and ${client} are the hostname of server and client. |
385385
| ib-traffic/ib\_write\_lat\_${msg_size}\_${direction}\_${line}\_${pair}:${server}\_${client} | time (us) | The max latency of perftest (ib_write_lat, ib_send_lat, ib_read_lat) using ${msg_size} with ${direction}('cpu-to-cpu'/'gpu-to-gpu'/'gpu-to-cpu'/'cpu-to-gpu') run between the ${pair}<sup>th</sup> node pair in the ${line}<sup>th</sup> line of the config, ${server} and ${client} are the hostname of server and client. |
386386

387+
### `nvbenchmark`
388+
389+
#### Introduction
390+
391+
Measures bandwidth and latency for various memcpy patterns across different links using copy engine or kernel copy methods.
392+
393+
#### Metrics
394+
395+
| Metrics | Unit | Description |
396+
|---------------------------------------------------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
397+
| host_to_device_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | Host to device CE memcpy using cuMemcpyAsync |
398+
| host_to_device_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
399+
| device_to_host_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | Device to host CE memcpy using cuMemcpyAsync |
400+
| device_to_host_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
401+
| host_to_device_bidirectional_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | A host to device copy is measured while a device to host copy is run simultaneously. Only the host to device copy bandwidth is reported. |
402+
| host_to_device_bidirectional_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
403+
| device_to_host_bidirectional_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | A device to host copy is measured while a host to device copy is run simultaneously. Only the device to host copy bandwidth is reported. |
404+
| device_to_host_bidirectional_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
405+
| device_to_device_memcpy_read_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of cuMemcpyAsync between each pair of accessible peers. Read tests launch a copy from the peer device to the target using the target's context. |
406+
| device_to_device_memcpy_read_ce_sum_bw | GB/s | Sum of the output matrix |
407+
| device_to_device_memcpy_write_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of cuMemcpyAsync between each pair of accessible peers. Write tests launch a copy from the target device to the peer using the target's context. |
408+
| device_to_device_memcpy_write_ce_sum_bw | GB/s | Sum of the output matrix |
409+
| device_to_device_bidirectional_memcpy_read_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of cuMemcpyAsync between each pair of accessible peers. A copy in the opposite direction of the measured copy is run simultaneously but not measured. Read tests launch a copy from the peer device to the target using the target's context. |
410+
| device_to_device_bidirectional_memcpy_read_ce_sum_bw | GB/s | Sum of the output matrix |
411+
| device_to_device_bidirectional_memcpy_write_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of cuMemcpyAsync between each pair of accessible peers. A copy in the opposite direction of the measured copy is run simultaneously but not measured. Write tests launch a copy from the target device to the peer using the target's context. |
412+
| device_to_device_bidirectional_memcpy_write_ce_sum_bw | GB/s | Sum of the output matrix |
413+
| all_to_host_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of cuMemcpyAsync between a single device and the host while simultaneously running copies from all other devices to the host. |
414+
| all_to_host_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
415+
| all_to_host_bidirectional_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | A device to host copy is measured while a host to device copy is run simultaneously. Only the device to host copy bandwidth is reported. All other devices generate simultaneous host to device and device to host interferring traffic. |
416+
| all_to_host_bidirectional_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
417+
| host_to_all_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of cuMemcpyAsync between the host to a single device while simultaneously running copies from the host to all other devices. |
418+
| host_to_all_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
419+
| host_to_all_bidirectional_memcpy_ce_cpu[0-9]_gpu[0-9]_bw | GB/s | A host to device copy is measured while a device to host copy is run simultaneously. Only the host to device copy bandwidth is reported. All other devices generate simultaneous host to device and device to host interferring traffic. |
420+
| host_to_all_bidirectional_memcpy_ce_sum_bw | GB/s | Sum of the output matrix |
421+
| all_to_one_write_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from all accessible peers to a single device, for each device. Bandwidth is reported as the total inbound bandwidth for each device. Write tests launch a copy from the target device to the peer using the target's context. |
422+
| all_to_one_write_ce_sum_bw | GB/s | Sum of the output matrix |
423+
| all_to_one_read_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from all accessible peers to a single device, for each device. Bandwidth is reported as the total outbound bandwidth for each device. Read tests launch a copy from the peer device to the target using the target's context. |
424+
| all_to_one_read_ce_sum_bw | GB/s | Sum of the output matrix |
425+
| one_to_all_write_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from a single device to all accessible peers, for each device. Bandwidth is reported as the total outbound bandwidth for each device. Write tests launch a copy from the target device to the peer using the target's context. |
426+
| one_to_all_write_ce_sum_bw | GB/s | Sum of the output matrix |
427+
| one_to_all_read_ce_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from a single device to all accessible peers, for each device. Bandwidth is reported as the total inbound bandwidth for each device. Read tests launch a copy from the peer device to the target using the target's context. |
428+
| one_to_all_read_ce_sum_bw | GB/s | Sum of the output matrix |
429+
| host_to_device_memcpy_sm_cpu[0-9]_gpu[0-9]_bw | GB/s | Host to device SM memcpy using a copy kernel |
430+
| host_to_device_memcpy_sm_sum_bw | GB/s | Sum of the output matrix |
431+
| device_to_host_memcpy_sm_cpu[0-9]_gpu[0-9]_bw | GB/s | Device to host SM memcpy using a copy kernel |
432+
| device_to_host_memcpy_sm_sum_bw | GB/s | Sum of the output matrix |
433+
| device_to_device_memcpy_read_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of a copy kernel between each pair of accessible peers. Read tests launch a copy from the peer device to the target using the target's context. |
434+
| device_to_device_memcpy_read_sm_sum_bw | GB/s | Sum of the output matrix |
435+
| device_to_device_memcpy_write_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of a copy kernel between each pair of accessible peers. Write tests launch a copy from the target device to the peer using the target's context. |
436+
| device_to_device_memcpy_write_sm_sum_bw | GB/s | Sum of the output matrix |
437+
| device_to_device_bidirectional_memcpy_read_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of a copy kernel between each pair of accessible peers. Copies are run in both directions between each pair, and the sum is reported. Read tests launch a copy from the peer device to the target using the target's context. |
438+
| device_to_device_bidirectional_memcpy_read_sm_sum_bw | GB/s | Sum of the output matrix |
439+
| device_to_device_bidirectional_memcpy_write_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of a copy kernel between each pair of accessible peers. Copies are run in both directions between each pair, and the sum is reported. Write tests launch a copy from the target device to the peer using the target's context. |
440+
| device_to_device_bidirectional_memcpy_write_sm_sum_bw | GB/s | Sum of the output matrix |
441+
| all_to_host_memcpy_sm_cpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of a copy kernel between a single device and the host while simultaneously running copies from all other devices to the host. |
442+
| all_to_host_memcpy_sm_sum_bw | GB/s | Sum of the output matrix |
443+
| all_to_host_bidirectional_memcpy_sm_cpu[0-9]_gpu[0-9]_bw | GB/s | A device to host bandwidth of a copy kernel is measured while a host to device copy is run simultaneously. Only the device to host copy bandwidth is reported. All other devices generate simultaneous host to device and device to host interferring traffic using copy kernels. |
444+
| all_to_host_bidirectional_memcpy_sm_sum_bw | GB/s | Sum of the output matrix |
445+
| host_to_all_memcpy_sm_cpu[0-9]_gpu[0-9]_bw | GB/s | Measures bandwidth of a copy kernel between the host to a single device while simultaneously running copies from the host to all other devices. |
446+
| host_to_all_memcpy_sm_sum_bw | GB/s | Sum of the output matrix |
447+
| host_to_all_bidirectional_memcpy_sm_cpu[0-9]_gpu[0-9]_bw | GB/s | A host to device bandwidth of a copy kernel is measured while a device to host copy is run simultaneously. Only the host to device copy bandwidth is reported. All other devices generate simultaneous host to device and device to host interferring traffic using copy kernels. |
448+
| host_to_all_bidirectional_memcpy_sm_sum_bw | GB/s | Sum of the output matrix |
449+
| all_to_one_write_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from all accessible peers to a single device, for each device. Bandwidth is reported as the total inbound bandwidth for each device. Write tests launch a copy from the target device to the peer using the target's context. |
450+
| all_to_one_write_sm_sum_bw | GB/s | Sum of the output matrix |
451+
| all_to_one_read_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from all accessible peers to a single device, for each device. Bandwidth is reported as the total outbound bandwidth for each device. Read tests launch a copy from the peer device to the target using the target's context. |
452+
| all_to_one_read_sm_sum_bw | GB/s | Sum of the output matrix |
453+
| one_to_all_write_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from a single device to all accessible peers, for each device. Bandwidth is reported as the total outbound bandwidth for each device. Write tests launch a copy from the target device to the peer using the target's context. |
454+
| one_to_all_write_sm_sum_bw | GB/s | Sum of the output matrix |
455+
| one_to_all_read_sm_gpu[0-9]_gpu[0-9]_bw | GB/s | Measures the total bandwidth of copies from a single device to all accessible peers, for each device. Bandwidth is reported as the total inbound bandwidth for each device. Read tests launch a copy from the peer device to the target using the target's context. |
456+
| one_to_all_read_sm_sum_bw | GB/s | Sum of the output matrix |
457+
| host_device_latency_sm_cpu[0-9]_gpu[0-9]_lat | µs | Host - device SM copy latency using a ptr chase kernel |
458+
| host_device_latency_sm_sum_lat | µs | Sum of the output matrix |
459+
| device_to_device_latency_sm_gpu[0-9]_gpu[0-9]_lat | µs | Measures latency of a pointer dereference operation between each pair of accessible peers. Memory is allocated on a GPU and is accessed by the peer GPU to determine latency. |
460+
| device_to_device_latency_sm_sum_lat | µs | Sum of the output matrix |
461+
387462

388463
## Computation-communication Benchmarks
389464

‎examples/benchmarks/nvbandwidth.py

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT license.
3+
4+
"""Micro benchmark example for nvbandwidth benchmark.
5+
6+
Commands to run:
7+
python3 examples/benchmarks/nvbandwidth.py
8+
"""
9+
10+
from superbench.benchmarks import BenchmarkRegistry, Platform
11+
from superbench.common.utils import logger
12+
13+
if __name__ == '__main__':
14+
context = BenchmarkRegistry.create_benchmark_context(
15+
'nvbandwidth',
16+
platform=Platform.CPU,
17+
parameters=(
18+
'--buffer_size 128 '
19+
'--test_cases 0,1,19,20 '
20+
'--skip_verification '
21+
'--disable_affinity '
22+
'--use_mean '
23+
'--num_loops 10'
24+
)
25+
)
26+
27+
benchmark = BenchmarkRegistry.launch_benchmark(context)
28+
if benchmark:
29+
logger.info(
30+
'benchmark: {}, return code: {}, result: {}'.format(
31+
benchmark.name, benchmark.return_code, benchmark.result
32+
)
33+
)

‎superbench/benchmarks/micro_benchmarks/nvbandwidth.py

+4-1
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,10 @@ def add_parser_arguments(self):
4141
type=str,
4242
default='',
4343
required=False,
44-
help='Specify the test case(s) to run, either by name or index. By default, all test cases are executed..',
44+
help=(
45+
'Specify the test case(s) to run, either by name or index. By default, all test cases are executed. '
46+
'Example: --test_cases 0,1,2,19,20'
47+
),
4548
)
4649

4750
self._parser.add_argument(

0 commit comments

Comments
 (0)