With the MTU set for the ethernet device to be 9000 (jumbo frames), I can only send 4012 bytes using ibv_ud_pingpong. Is there a reason why its 4012?
I understand there are ethernet headers to be added with the payload but I though they were 40 bytes for a UD QP.
If I send a 4096-byte sized packet on a RC QP I see that the PSN only gets incremented once.
What am I missing here with regard to UD?
The only reference I found for the ibv_ud_pingpong and 4012 bytes was here: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/6.5_Technical_Notes/
I'm using ConnectX-3 HCAs on Ubuntu 14.04-based hosts connected back-to-back.