Make Infiniband counters visible inside worker pods

[rdjjke]

Soperator pods currently use hostNetwork: false. Because of that Infiniband counters are not visible containers' sysfs (/sys/class/infiniband/mlx5_0/ports/1/counters).

To make them visible there, we can try to install the RDMA CNI plugin to K8s.

[Uburro]

k8snetworkplumbingwg/sriov-network-device-plugin#345 (comment)

[thien-lm]

Hi @rdjjke @Uburro
does this make slurmd pod can not use infiniband ? i am wondering.

[asteny]

@thien-lm No, pod can use infiniband, everything fine. It's just about counters in /sys/class/infiniband/mlx5_0/ports/1/counters. That's useful metric, but it's not visible inside pod container.

[Uburro]

@thien-lm We do not support RDMA-CNI yet, as well as some features from this CNI. For example, the scheduling spec in requests and, as @asteny mentioned, the visibility of certain parameters.

[thien-lm]

Thanks for your explaination.
as @Uburro said, soperator currently does not support RDMA-CNI, so slurm worker pod can not use RDMA feature, am i right ?