I tried to upgrade the kernel to 3.16.46 ( MLNX_OFED_LINUX-3.4-126.96.36.199 ) , but it did not work.
I tried to upgrade the driver to MLNX_OFED_LINUX-4.1-188.8.131.52 (kernel: 3.16.46), but The kernel module (xprtrdma) does not seem to match the driver version.
alkx Can you help me and give me some tips ? Thank you
Thank you for reaching out to the Mellanox Support Community and for your patience in this matter.
Unfortunately, as from Mellanox OFED version 3.4-x and above NFSoRDMA is not supported anymore. This was never mentioned in the Release Notes for version 3.4-184.108.40.206 but it is mentioned in later released version.
If you want to use NFSoRDMA, you need to downgrade the driver to Mellanox OFED version 3.3-220.127.116.11 in which we do still support NFSoRDMA and is available for Ubuntu 14.04 or switch to the OS-vendor supplied driver (INBOX Driver) for the Mellanox NIC.
Hopefully this will clarify and resolves your issue.
Hi Martijn :
thank you very much for your reply
After downgraded the driver to Mellanox OFED version 3.3-18.104.22.168 from your advice, I have recompiled the mlnx-ofed-kernel module with nfsrdma support .
But still get same error "svcrdma: Error -12 posting RDMA_READ" while executing “fio --rw=randwrite“.
I wonder if there is a bug or some wrong configuration .
Hope to get any of your suggestions .
options taken from /etc/infiniband/info + --with-nfsrdma-mod
mlnx-ofed-kernel-3.3# ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod --with-mlx4_en-mod --with-mlx5-mod --with-ipoib-mod --with-sdp-mod --with-e_ipoib-mod --with-nfsrdma-mod
copy xprtrdma.ko to nfs client and svcrdma.ko to nfs server
mlnx-ofed-kernel-3.3# cp net/sunrpc/xprtrdma/xprtrdma.ko /lib/modules/3.16.46-031646-generic/updates/dkms/
mlnx-ofed-kernel-3.3# depmod -a && modprobe xprtrdma