Wireshark support for NVMe over Fabrics

Version 3

    This post discusses Wireshark support for NVMe over Fabrics (NVMe-oF). Wireshark support for NVMe-oF (over RDMA) was added in January 2017 to the master branch, and can be used starting with Version 2.4.0rc1.






    1. Install Wireshark version  2.40rc1 or later.



    2. Load an NVMe-oF wireshark recording (*.pcap, *.pcapng) file.



    Packet Format


    1. Here are the expected headers seen on the wire for NVMe over RDMA over UDP/IP.


    • Ethernet
    • IP
    • UDP
    • InfiniBand - this header includes the BTH, and may have other sub-headers such as RETH, IETH.
    • NVMe-oF - this header encapsulates the  NVMe command capsule.
    • NVMe





    2. Once Wireshark is opened, start NVMe-oF recording. Use the Filter options to filter the traffic to nvme or nvme-rdma headers.




    3. Here is an NVMe read command example. Use the filter nvme.cmd.opc ==2 for NVMe read.



    4. Attached are two Wireshark recordings, one that has all the headers (NVMe over RDMA over UDP/IP), and another  with only InfiniBand headers (different setups).