Skip to content

Commit 15e7365

Browse files
@FIR-757: Update SDK to 0.1.4 and update release to 0.0.3 for tsi-ggml (#20)
The test results with ./run_llama_cli.sh with 5 tokens is as follows +++ root@agilex7_dk_si_agf014ea:/usr/bin/tsi/v0.1.1.tsv31_06_06_2025/bin# ./run_llama_cli.sh my cat's name is Max. He' llama_perf_sampler_print: sampling time = 111.70 ms / 11 runs ( 10.15 ms per token, 98.47 tokens per second)llama_perf_context_print: load time = 132926.48 ms llama_perf_context_print: prompt eval time = 109957.33 ms / 6 tokens (18326.22 ms per token, 0.05 tokens per second) llama_perf_context_print: eval time = 195682.91 ms / 4 runs (48920.73 ms per token, 0.02 tokens per second) llama_perf_context_print: total time = 328764.01 ms / 10 tokens GGML Tsavorite Profiling Results: ------------------------------------------------------------------------------------------------------------------------ Calls Total(ms) T/call Self(ms) Function ------------------------------------------------------------------------------------------------------------------------ 33160 100086.000 3.018 47907.157 [32%] RuntimeHostShim::awaitCommandListCompletion 18920 29912.952 1.581 29912.952 └─ [10%] [ txe_silu ] 14080 22010.102 1.563 22010.102 └─ [ 7%] [ txe_mult ] 160 253.071 1.582 253.071 └─ [ 0%] [ txe_add ] 33160 1.178 0.000 1.178 └─ [ 0%] TXE 0 Idle 1 114.000 114.000 18.000 [ 0%] GGML Tsavorite 1 96.000 96.000 96.000 └─ [ 0%] RuntimeHostShim::initialize 1 52.000 52.000 52.000 [ 0%] RuntimeHostShim::finalize 33160 26.000 0.001 26.000 [ 0%] RuntimeHostShim::loadBlob 33160 23.000 0.001 23.000 [ 0%] RuntimeHostShim::finalizeCommandList 33160 5.000 0.000 5.000 [ 0%] RuntimeHostShim::addCommandToList 33161 3.000 0.000 3.000 [ 0%] RuntimeHostShim::allocate 33160 3.000 0.000 3.000 [ 0%] RuntimeHostShim::createCommandList 113720 0.000 0.000 0.000 [ 0%] RuntimeHostShim::getShmemManager 33160 0.000 0.000 0.000 [ 0%] RuntimeHostShim::launchBlob 33160 0.000 0.000 0.000 [ 0%] RuntimeHostShim::unloadBlob 33160 0.000 0.000 0.000 [ 0%] RuntimeHostShim::deallocate ======================================================================================================================== 412163 308849.000 0.749308849.000 [100%] TOTAL ======================================================================================================================== root@agilex7_dk_si_agf014ea:/usr/bin/tsi/v0.1.1.tsv31_06_06_2025/bin# +++
1 parent f5713b3 commit 15e7365

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

tsi-pkg-build.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ echo 'updating submodule'
66
git submodule update --recursive --init
77
cd ggml-tsi-kernel/
88
module load tsi4 gcc/13.3.0
9-
export MLIR_SDK_VERSION=/proj/rel/sw/sdk-r.0.1.3
9+
export MLIR_SDK_VERSION=/proj/rel/sw/sdk-r.0.1.4
1010
echo 'creating python virtual env'
1111
/proj/local/Python-3.10.12/bin/python3 -m venv blob-creation
1212
source blob-creation/bin/activate
@@ -48,7 +48,7 @@ cmake --build build-fpga --config Release
4848

4949

5050
echo 'creating tar bundle for fpga'
51-
TSI_GGML_VERSION=0.0.2
51+
TSI_GGML_VERSION=0.0.3
5252
TSI_GGML_BUNDLE_INSTALL_DIR=tsi-ggml
5353
GGML_TSI_INSTALL_DIR=ggml-tsi-kernel
5454
TSI_GGML_RELEASE_DIR=/proj/rel/sw/ggml/

0 commit comments

Comments
 (0)