We are using Mellanox ConnectX-3 Pro cards on Windows Server 2016 with WinOF-5.35 driver installed. We have written a sample NDKCI driver to send IOs using MS provided NDK APIs.
Before sending any IOs, we are creating FRMR for the buffer and Unqiue key for the same. Using NDKSend() we are sending the capsule to the target, further Provider will initate
callbacks which were registerd earlier for Send and Receive completion. This above sequence works very well when we send IOs synchronously.
But if we send IOs asynchronously, Sometime NDKSend() gives success status but we are not getting Send or Receive callbacks. Even from the Ibdump we could not see the capsule sent from the Host.
But Same sequence works very well with Normal MR instead of FRMR in both Sync and Async.
We wanted to know, Do we need to do specially anything for FRMR in Async? or Does Mellanox have any open issue with respect to FRMR?
Thanks in advance,