Skip to content

Commit db4edea

Browse files
kaiyuxbloodeagle40234Pzzzzz5142pathorn
authoredJun 11, 2024
Update TensorRT-LLM (NVIDIA#1763)
* Update TensorRT-LLM --------- Co-authored-by: Kota Tsuyuzaki <bloodeagle40234@gmail.com> Co-authored-by: Pzzzzz <hello-cd.plus@hotmail.com> Co-authored-by: Patrick Reiter Horn <patrick.horn@gmail.com>
1 parent b777bd6 commit db4edea

File tree

301 files changed

+556832
-523784
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

301 files changed

+556832
-523784
lines changed
 

‎benchmarks/cpp/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ mpirun -n 2 ./benchmarks/gptManagerBenchmark \
159159
--max_num_samples 500
160160
```
161161

162-
`gptManagerBenchmark` can also be used with the high-level C++ API defined by the `executor::Executor` class (see `cpp/include/tensorrt_llm/executor/executor.h`). This can be done by passing the argument `--api executor`. Note that the Executor class is still under development and currently does not support models with tp or pp > 1.
162+
`gptManagerBenchmark` by default uses the high-level C++ API defined by the `executor::Executor` class (see `cpp/include/tensorrt_llm/executor/executor.h`).
163163

164164
#### Emulated static batching
165165

‎benchmarks/cpp/gptManagerBenchmark.cpp

+86-77
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)