I'm test Mellanox solutions that include Mellanox MCX354A-FCBT ConnectX-3 HCAs with firmware 2.42.5000, Mellanox FDR14 2m copper cables, Mellanox SX6036G Gateway switches with firmware 3.6.4006 on vSphere 6.5 update 1 for POC.
It'll be going with 3 steps like belows.
01. Build a initial network connection with TCP/IP (lossy) network
02. Build a PFC network
03. Build a iSER RDMA Storage
Here are POC systems lists.
* ESXi hosts
: 5 x Fujitsu RX200 S7 ESXi Hosts that include 2Way E5-2670, 128GB Memory, RAID1 SSD boot disk for ESXi, Mellanox ConnectX-3 MCX354A-FCBT
* Ethernet Switch
: 2 x Mellanox SX6036G Gateway Systems as a PFC Ethernet switch that support 1, 10, 40, 56Gb Ethernet port mode
: 5 x SuperMicro Custom Storage with ESOS SCST Target - 1Way E5-2660, 64GB Memory, Mellanox ConnectX-3 MC354A-FCBT
I want to build PFC based infrastructures that include
01.General purpose VM ETH network with TC0 that lossy ETH network
02. Build a PFC ETH network
02-A. PFC based iSER RDMA Storage ETH network with TC1 that loseless ETH network
03-B. PFC based vMotion RDMA ETH network with TC02 that leseless ETH network
03. General purpose vKernel NFS network for NFS Storage with TC04 that lossy ETH network
At first, some times later I'll post step 01 results.
Anybody who interesting this POC test, participate this threads...
I gave up this project.
Mellanox deprecate ConnectX-3 & Connect-3 Pro products.
ConnectX-3 will be deprecate on 1 Dec., 2017.
ConnectX-3 Pro will be deprecate on 31 Jan., 2018.
I predict new iSER driver on ESXi 6.5 & next relase ESXi 6.7 will be support ConnectX-4 or above products.
ConnectX-4 OCP product also deprecated.
In worst case, I think move to ConnectX-5 or ConnectX-6 product.