Note sure if this is any help:
Have you tried to file it as a bug to the Intel MPI team?
Does the same thing happen if you use another MPI implementation?
MOFED does have openmpi that use UD QP by the default, or you can try HPC-X Toolkit from Mellanox site, that available for different OSes.
Check ulimits (Environment Problems | Intel® Developer Zone ) and maybe limits.conf