I enabled SR-IOV on my card, I see 5 new infiniband cards but only one has a port GUID. Is there a way to fix this?
With this current configuration I can pass one of the virtual interface (ib1) to a VM but I can't send ARP request from a KVM VM (I can from ib0 on the host). Do I need some kind of bridging there?
I'm on ubuntu 13.0.4, can I just use a stock kernel to configure my Mellanox? OFED and Mellanox drivers do not work on this distro.
Could you please describe your full stack for your SR-IOV implementation?
Also, "but I can't send ARP request from a KVM VM"
Unless there is a special feature I am unaware of. I would assume this is not possible as it is in the ib0-host is because the virtualization pushes up at the PCI level, breaking some upper layer mappings.