What is RDMA?

Version 34

    This post is aimed for beginners that wish to understand basic InfiniBand RDMA concepts.

     

    >> Learn  RDMA on the Mellanox Academy for free

     

    References

     

     

    Direct Memory Access (DMA) is an ability of a device to access host memory directly, without the intervention of the CPU.

    Remote Direct Memory Access (RDMA) is the ability of accessing (read, write) memory on a remote machine without interrupting the processing of the CPU(s) on that system.

     

    Here is a short video that explains it:

     

    RDMA Advantages

     

    • Zero-copy - applications can perform data transfers without the involvement of the network software stack. Data is sent and received directly to the buffers without being copied between the network layers.

    • Kernel bypass - applications can perform data transfers directly from user-space without kernel involvement.

    • No CPU involvement - applications can access remote memory without consuming any CPU time in the remote server. The remote memory server will be read without any intervention from the remote process (or processor). Moreover, the caches of the remote CPU will not be filled with the accessed memory content.

     

     

    RDMA Solution Markets

     

    RDMA is being used in many industries. Some example are:

    • High Performance Computing (HPC)
    • Data Centers
    • Web 2.0
    • Big Data
    • Cloud
    • Storage
    • Finance Services and Insurance (FSI)
    • Medical Services
    • Federal

     

    Refer to the Solutions page on www.mellanox.com for more information.

     

    How Do I Use RDMA?

    In order to use RDMA, you need a network adapter that has RDMA capability (such as Mellanox's Connect-X family of adapters).

    The link layer protocol of the network can be either Ethernet or InfiniBand -- both can transfer the RDMA based applications.

    RDMA is supported in-box on the operating systems Linux, Windows, and VMware. Other Operating Systems (or for advanced features) you may need to download and install the relevant driver package and configure it accordingly.

     

    RDMA Implementation Examples:

     

    - Storage RDMA Protocol (SRP)

    - iSCSI Extensions for RDMA (iSER)

    - Windows SMB Direct

    - Lustre (File System)

    - IBM GPFS (Platform Computing)

    - IPoIB

    - Apache Hadoop (UDA & R4H)

    - Gluster

    - Ceph

    - NFSoRDMA

    - NVMe over Fabrics (under development as a standard)

     

    What is RoCE?

    RoCE stands for RDMA over Converged Ethernet. It is a standard that enables passing RDMA traffic over an Ethernet network.

     

    More information can be found via the references above.

     

    Use Cases