threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling #12995
+29
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
examples
ggml
max-krasnyansky
force-pushed
the
maxk/sched-prio-updates
branch
2 times, most recently
from
bced6ad to
b4458d4
Compare
slaren
reviewed
Apr 17, 2025
max-krasnyansky
force-pushed
the
maxk/sched-prio-updates
branch
from
a157d1b to
854252f
Compare
max-krasnyansky
force-pushed
the
maxk/sched-prio-updates
branch
from
854252f to
9d4b917
Compare
…dows to avoid throttling We talked about adding LOW priority for GGML threads in the original threadpool PR. It might be useful for some cases to avoid contention. Latest Windows ARM64 releases started parking (offlining) the CPU cores more aggresively which results in suboptimal performance with n_threads > 4. To deal with that we now disable Power Throttling for our threads for the NORMAL and higher priorities. Co-authored-by: Diego Devesa <[email protected]>
max-krasnyansky
force-pushed
the
maxk/sched-prio-updates
branch
from
9d4b917 to
f155d23
Compare
|
The CI failure here is a bit weird.. it is actually using mingw rather than MSVC, and the problem seems to be that this version of mingw does not support this API at all. We should just remove the msvc-arm64 build from the CI, along with the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We talked about adding LOW priority for GGML threads in the original threadpool PR. It might be useful for some cases to avoid contention.
Latest Windows ARM64 releases started parking (offlining) the CPU cores more aggresively which results in suboptimal performance with n_threads > 4. To deal with that we now disable Power Throttling for our threads for the NORMAL and higher priorities.
Make sure to read the contributing guidelines before submitting a PR