Parallelizing POMDPs.jl code on GPU

[Manavvora]

Hi,

I have defined a POMDP using the QuickPOMDPs interface and am solving it using the BasicPOMCP solver. My code is easily parallelizable as I am doing the same procedure for different values sequentially. However, when I tried writing a GPU kernel for the same, I got lots of errors of the form : unsupported call through a literal pointer (call to ijl_alloc_string), unsupported call through a literal pointer (call to ijl_string_to_array), etc.

How do I solve this error? I am currently using colab for running this code.

[zsunberg]

Hmm... we haven't tried too much of POMDPs.jl out on GPUs, but QuickPOMDPs objects are probably not very suitable for it since the data structure itself wraps a bunch of functions. It should, however, be possible to run a POMDP model defined using the object-oriented method (https://juliapomdp.github.io/POMDPs.jl/stable/def_pomdp/#Object-oriented) on a GPU.

Are you actually trying to run POMCP on a GPU or are you just trying to simulate the POMDP or do something simpler? The error messages seem to suggest that your kernel is trying to allocate memory. I do not think it is possible to run the current BasicPOMCP solver in a GPU kernel since it allocates memory.

Let us know what you're trying to accomplish on the GPU, and we can help you figure out how to move forward!

[rejuvyesh]

It's unlikely that any of the POMDP solvers would work out of the box on a GPU and would likely require custom kernels for actual speedups.

[rejuvyesh]

Can you clarify what you mean by trying to write a GPU kernel? Is there a forked codebase we can look at?

[Manavvora]

It's something like this:

`function gpu_add1!(y, x)
    for i = 1:length(y)
        @inbounds y[i] += x[i]
    end
    return nothing
end

fill!(y_d, 2)
@cuda gpu_add1!(y_d, x_d)
@test all(Array(y_d) .== 3.0f0)`

But instead of the y[i] += x[i] I want to add my own function which defines a POMDP model using QuickPOMDP and solves it using BasicPOMCP.

[zsunberg]

My knowledge of GPU technology may be a bit out of date, but I don't think this is really what they are designed to be used for. GPUs are typically used to perform the same very simple calculations simultaneously on an array of data. So anything that involves memory allocation, or even branching is not what they are designed to do. I don't think it will be possible to run a POMDP solver in an individual GPU kernel.

A much better use would be to use GPU-computing within a POMDP solver to carry out repeated operations quickly. This would require writing new code, but if you are interested in pursuing that, we can explain further.

[Manavvora]

For now, I am just parallelizing it on the CPU but thank you for your input. I'll get back to you when I begin pursuing the GPU path.