1 Reply Latest reply on Oct 1, 2017 6:10 AM by march

    Is VMA really use RDMA to implement offload?


      Hello, I'm learning RDMA and testing VMA. In the VMA user manual, it says that VMA implements native RDMA verbs API. And in the source code of VMA, I really see the RDMA verbs. However, when I run VMA with LD_PRELOAD and VMA_SPEC=latency, it runs slower than TCP. I add a log before the ibv_post_send function invocation and recompile the code. The log does not show up! And in the source code, I see the enum type below


      enum tcp_sock_offload_e {

      TCP_SOCK_PASSTHROUGH = 1, // OS handling this socket connection

      // TCP_SOCK_RDMA_CM,         // Offloaded, uses RDMA CM - SDP like connection

      TCP_SOCK_LWIP             // Offloaded, uses LWIP for wire compatible TCP impl



      The TCP_SOCK_RDMA_CM is annotated! But it is also true that VMA implements the RDMA verbs. I am confused that which VMA really uses to implement offload, RDMA or LWIP? Or do I miss some configuration parameters that are necessary when using RDMA in VMA?


      I test the VMA on the machine with ConnectX®-5 and the OFED 4.1 is installed.

        • Re: Is VMA really use RDMA to implement offload?



          The enum you see, just means that for RDMA_CM (RDMA connection manager), VMA is not used (commented).The establishment of the QP will not pass through the stack.

          So the two options for VMA is either the stack handles the connection or the VMA (that uses the lightweight IP stack).


          If you want to understand what is the rdma cm , run ib_write_bw like example with -R option, the rdma cm will handle the connection to exchange QP attributes like example.