[cudadev][RFC] Prototype (host|device)_unique_ptr API to use lightweight "Context" object instead of CUDA stream #256
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add runAcquire(), runProduce() functions
… in better-defined way
makortel
commented
Oct 26, 2021
Comment on lines
+8
to
+30
| class HostAllocatorContext { | ||
| public: | ||
| explicit HostAllocatorContext(cudaStream_t stream) : stream_(stream) {} | ||
|
|
||
| void *allocate_host(size_t nbytes) const { return cms::cuda::allocate_host(nbytes, stream_); } | ||
|
|
||
| void free_host(void *ptr) const { cms::cuda::free_host(ptr); } | ||
|
|
||
| private: | ||
| cudaStream_t stream_; | ||
| }; | ||
|
|
||
| class DeviceAllocatorContext { | ||
| public: | ||
| explicit DeviceAllocatorContext(cudaStream_t stream) : stream_(stream) {} | ||
|
|
||
| void *allocate_device(size_t nbytes) const { return cms::cuda::allocate_device(nbytes, stream_); } | ||
|
|
||
| void free_device(void *ptr) const { cms::cuda::free_device(ptr, stream_); } | ||
|
|
||
| private: | ||
| cudaStream_t stream_; | ||
| }; |
Collaborator
Author
There was a problem hiding this comment.
Right now (and possibly forever in cudadev) the HostAllocatorContext and DeviceAllocatorContext look nearly identical, but in the future (in CMSSW) they could hold a pointer to the CachingHostAllocator/CachingDeviceAllocator objects.
…uda::Context objects instead of cudaStream_t
makortel
force-pushed
the
cudadev_next_api_v3_allocator_via_context_v3
branch
from
f2054b5 to
c91e0e3
Compare
5 tasks
Collaborator
Author
|
Made effectively obsolete by cms-sw/cmssw#39428 (although this particular development is not part of the CMSSW PR). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR builds on top of #224, but because of the actual developments conflict between the base commit of #224 and
master, also the #224 part is rebased. The actual developments of this PR are in the last three commits.The change can be summarized in
make_device_unique<T>(stream)changing tomake_device_unique(ctx)wherectxcan be e.g. theAcquireContext/ProduceContext, or a "lightweight"HostAllocatorContext/DeviceAllocatorContext/Context(theAcquireContext/ProduceContextare convertible to the latterContextobjects). (I'm really overusing the "Context" term here, but haven't figured out better wording yet).The idea is that
HostAllocatorContextprovides the access to pinned host memory allocator (and only that)DeviceAllocatorContextprovides access to device memory allocator (and only that)Contextprovides access to both pinned host and device memory allocators (via conversions to the two former types), and also whatever is needed to launch asynchronous kernels or memory transfers (in practice the CUDA stream)This change would allow e.g.
CachingDeviceAllocatorandCachingHostAllocatorobjects from global variables to be owned (again) byCUDAService(in CMSSW only), that would further enable (again) the caching allocator parameters be configured at run timeAcquireContext/ProduceContextfor better performance (see discussion in [cudadev] Improve caching allocator performance #218)