Add internal cuda::__is_device_memory

[fbusato]

Description

PR #6733 shows that we also need a relaxed cuda::is_device_accessible which doesn't require a device reference. The overall idea is to just check if a pointer is device allocated.

[davebayer]

This version should be internal only, trying to be optimistic - if something fails, let's just return true.

[fbusato]

Why it should be internal only? It looks consistent and aligned with the other APIs. If we need it, it is very likely that it could be used by other users as well.

if something fails, let's just return true.

I don't understand. The behavior is roughly equivalent to is_device_accessible(ptr, id) for each id.

[davebayer]

Because we cannot ask this question in general. If there are 2 devices without peer access enabled, you cannot possibly know the memory living in 1st device is not accessible from the second, so what should the function return in that case?

Also, I disagree that it is well aligned with what we do in other apis. All device apis take either explicit device_ref parameter or get the device from stream for example. If you have a device memory resource, you specify which device you want to create the resource for.

If we must expose this, is_any_device_accessible(ptr) would be better in my opinion. But I still think that we should make the function internal so we can just return true in case of an error, because I believe we don't want the device accessor to throw a cuda_error if we are just not able to confirm it really is accessible from a device.

[fbusato]

Because we cannot ask this question in general. If there are 2 devices without peer access enabled, you cannot possibly know the memory living in 1st device is not accessible from the second, so what should the function return in that case?

my point is not if it is accessible from GPU A or B. The information that I get is that the input is a device pointer, which is already something valuable.

Also, I disagree that it is well aligned with what we do in other apis.

I meant consistent with other is_*_accessible API. There are situations where the device id is not available as well.

If we must expose this, is_any_device_accessible(ptr) would be better in my opinion.

We can keep it internal if it creates ambiguity, I'm not strongly against it. However, I'm more in favor of exposing it. Peer accessibility looks more an edge case (I could be wrong).

[pciolkosz]

Could we name it is_device_memory? Or something similar? I really think it would be giving a wrong message and someone might do is_device_accessible(ptr) -> access ptr from wrong device, get a illegal memory access and wonder why the check was wrong. It might look like a is_device_accessible(ptr, current_device) or give impression that accessibility doesn't need to take the device into account. Which is why I think a different name would fit better, because it wouldn't imply it gives a definitive answer to accessibility.
Internal only is also an option, because then you never give the wrong impressions I listed above

[fbusato]

is_device_memory is a better name for sure and expresses the intent of the function more accurately.

[fbusato]

ok, let keep it internal. I need it to unblock a chain of PRs.

[github-actions]

🥳 CI Workflow Results

🟩 Finished in 1h 37m: Pass: 100%/91 | Total: 1d 18h | Max: 1h 37m | Hits: 96%/201901

See results here.

[davebayer]

I think it's counterintuitive that __is_device_memory includes managed memory as well

[fbusato]

I really open to suggestions for a better name. The idea of the function is to provide a weak guarantee about device memory access when the device id is not available. __is_device_accessible has been discarded because of peer access