Create TT client via plugin for TensTorrent devices #1860
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
giordano
force-pushed
the
mg/tt-plugin
branch
from
91e0d7c to
d6da889
Compare
giordano
force-pushed
the
mg/tt-plugin
branch
2 times, most recently
from
4e7fde2 to
72852fc
Compare
src/accelerators/TT.jl
Outdated
| end | ||
| end | ||
|
|
||
| has_tt() = true |
There was a problem hiding this comment.
I need to figure out how to detect the devices automatically, but apart from that this should be good from my side as an experimental plugin. I'm not sure about the "TT" name everywhere though, bit too short and obscure, but that's how all the TensTorrent tools are called.
There was a problem hiding this comment.
Reminder to self: one way to check this is looking for the /dev/tenstorrent/ directory
There was a problem hiding this comment.
Alternatively, similarly to the TPU discovery, 0x1e52 is the Tenstorrent vendor ID in /sys/bus/pci/devices:
julia> [dir for dir in readdir("/sys/bus/pci/devices"; join=true) if readchomp(joinpath(dir, "vendor")) == "0x1e52"]
32-element Vector{String}:
"/sys/bus/pci/devices/0000:01:00.0"
"/sys/bus/pci/devices/0000:02:00.0"
"/sys/bus/pci/devices/0000:03:00.0"
"/sys/bus/pci/devices/0000:04:00.0"
"/sys/bus/pci/devices/0000:05:00.0"
"/sys/bus/pci/devices/0000:06:00.0"
"/sys/bus/pci/devices/0000:07:00.0"
"/sys/bus/pci/devices/0000:08:00.0"
"/sys/bus/pci/devices/0000:41:00.0"
"/sys/bus/pci/devices/0000:42:00.0"
"/sys/bus/pci/devices/0000:43:00.0"
"/sys/bus/pci/devices/0000:44:00.0"
"/sys/bus/pci/devices/0000:45:00.0"
"/sys/bus/pci/devices/0000:46:00.0"
"/sys/bus/pci/devices/0000:47:00.0"
"/sys/bus/pci/devices/0000:48:00.0"
"/sys/bus/pci/devices/0000:81:00.0"
"/sys/bus/pci/devices/0000:82:00.0"
"/sys/bus/pci/devices/0000:83:00.0"
"/sys/bus/pci/devices/0000:84:00.0"
"/sys/bus/pci/devices/0000:85:00.0"
"/sys/bus/pci/devices/0000:86:00.0"
"/sys/bus/pci/devices/0000:87:00.0"
"/sys/bus/pci/devices/0000:88:00.0"
"/sys/bus/pci/devices/0000:c1:00.0"
"/sys/bus/pci/devices/0000:c2:00.0"
"/sys/bus/pci/devices/0000:c3:00.0"
"/sys/bus/pci/devices/0000:c4:00.0"
"/sys/bus/pci/devices/0000:c5:00.0"
"/sys/bus/pci/devices/0000:c6:00.0"
"/sys/bus/pci/devices/0000:c7:00.0"
"/sys/bus/pci/devices/0000:c8:00.0"Looking at /dev/tenstorrent should be good as well, that's the kind of thing JAX does for Nvidia GPUs anyway: https://github.com/jax-ml/jax/blob/152099ee0ef31119f16f4c2dac50d84fcb1575ef/jax/_src/hardware_utils.py#L35-L39
|
Let's add it to https://github.com/EnzymeAD/Reactant.jl/blob/mg/tt-plugin/docs/src/index.md#select-an-accelerator-backend as well (maybe with |
giordano
force-pushed
the
mg/tt-plugin
branch
from
c819a82 to
e78b887
Compare
|
Ok, I think this is ready for review now. |
| Reactant.set_default_backend("tpu") | ||
| ``` | ||
|
|
||
| ```julia [Tenstorrent (Experimental)] |
There was a problem hiding this comment.
while updating this do we want to mention AMD gpus
There was a problem hiding this comment.
How do you set the backend? Is it just gpu like for Nvidia? Also, does it actually work? At least this backend can do a matmul 😛
There was a problem hiding this comment.
In principle it should just also be GPU
Based on the template of #1626. This is very preliminary, but very basic stuff works if we ignore the result of a matmul is off by ~0.1%, which doesn't look terribly good (accuracy is a lot better on CPU):
Ref: #1256, #1235 (CC @p-w-rs)
@fleclairTT you might be interested in this, too.