I downloaded the latest OFED software and the nvidia peer memory driver. The PEER_MEMORY_API.txt file references a call to ib_register_peer_memory_client. I cannot find this call in any of the header or source files in the OFED distribution. Where is that function defined? Also, is there any example code showing how to properly use it? The goal is to use ibverbs calls to transfer data directly from a GPU to another system via Infiniband. Is the PEER_MEMORY_API the way to do this?
I figured out how to use the API. Thanks to the new OFED code from Mellanox, you only have to pass the CUDA malloc'ed address to the infiniband memory register call. If you have the nvidia_peer_memory driver from Mellanox installed, that driver is called to validate the address.