5 Replies Latest reply on Mar 15, 2018 11:35 AM by ballle_98

    Problem with vlan with multicast mac over SR-IOV (VMware)

    robertw

      Hi,

       

      I have big problem with multicast mac with vlan tag. I suspect filtering and droping packet somewhere within VMware.

      Set in VMware VLAN = 4095 for SR-IOV. Tests in both as a mode PCI Device and BYPASS mode (through Port Group).

       

      Problem does not occur when the cards are configured in the passthrough mode (no SR-IOV mode).

      Multicasts not tagged, pass without a problem (Precison Time Protocol).

       

      Running tcpdump on the side of the sender log:

      Frame 1: 144 bytes on wire (1152 bits), 144 bytes captured (1152 bits)

          Encapsulation type: Ethernet (1)

          Arrival Time: Mar  4, 2018 21:35:15.204041000 Central European Standard Time

          [Time shift for this packet: 0.000000000 seconds]

          Epoch Time: 1520195715.204041000 seconds

          [Time delta from previous captured frame: 0.000000000 seconds]

          [Time delta from previous displayed frame: 0.000000000 seconds]

          [Time since reference or first frame: 0.000000000 seconds]

          Frame Number: 1

          Frame Length: 144 bytes (1152 bits)

          Capture Length: 144 bytes (1152 bits)

          [Frame is marked: False]

          [Frame is ignored: False]

          [Protocols in frame: eth:ethertype:vlan:ethertype:sv]

          [Coloring Rule Name: Broadcast]

          [Coloring Rule String: eth[0] & 1]

      Ethernet II, Src: Vmware_51:c4:8e (00:0c:29:51:c4:8e), Dst: Iec-Tc57_04:00:00 (01:0c:cd:04:00:00)

          Destination: Iec-Tc57_04:00:00 (01:0c:cd:04:00:00)

              Address: Iec-Tc57_04:00:00 (01:0c:cd:04:00:00)

              .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)

              .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)

          Source: Vmware_51:c4:8e (00:0c:29:51:c4:8e)

              Address: Vmware_51:c4:8e (00:0c:29:51:c4:8e)

              .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)

              .... ...0 .... .... .... .... = IG bit: Individual address (unicast)

          Type: 802.1Q Virtual LAN (0x8100)

      802.1Q Virtual LAN, PRI: 7, DEI: 0, ID: 4

          111. .... .... .... = Priority: Network Control (7)

          ...0 .... .... .... = DEI: Ineligible

          .... 0000 0000 0100 = ID: 4

          Type: IEC 61850/SV (Sampled Value Transmission (0x88ba)

      IEC61850 Sampled Values

          APPID: 0x4000

          Length: 126

          Reserved 1: 0x0000 (0)

          Reserved 2: 0x0000 (0)

          savPdu

              noASDU: 1

              seqASDU: 1 item

       

      In this document, http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_ESXi_User_Manual_v2.4.0.pdf. Point  3.3.1.5 is written:

      "Any vlan-tagged packets sent by the VF are silently dropped."

       

      Configuration:

      Server: 3 x Dell  PowerEdge R710 rev II (12 CPU - 24 logical, 96GB RAM)

      Mellanox Card: 3 x ConnectX-4 Lx EN-MCX4121A-XCAT (latest FW 14.21.2010,  PXE 3.5.0305)

      Switches: 2 x Cisco Nexus 3048TP-1GE (latest stable firmware 7.0.3.I7.3)

       

      VMware: DellEMC-ESXi-6.5U1-7388607-A07 (latest from Dell)

      VM OS: CentOS Linux release 7.4.1708 (Core, updated)

      VMware network driver: nmlx5_core (VMware native, 4.16.10.3-1OEM.650.0.0.4598673)

      VM OS network driver mlx5_core: 4.2-1.0.1 (latest Linux driver EN)

       

      How to solve the problem, otherwise research does not make much sense.

      I am asking for support, it is very important for Me and My thesis, research.

       

       

      Best Regards,

      Robert