1 Reply Latest reply on May 16, 2018 8:05 PM by yairi

    libi40iw-i40iw_upost_send: post work request failed, invalid opcode: 0x3

    mabentele

      hello

      running an  mpi test on ib Fails.

      host is hpe xl230 redhatel 7.5 mlx5 . ibstat ibping is successful. Limits memlock are unlimited

       

       

      any ideas?

      thanks a lot

      Markus Bentele

       

      starccm+ -licpath 1716@flexlmhpc -mpi platform -fabricverbose -mppflags "-v -prot -e MPI_IC_ORDER=ibv -affopt=v" -on fnx601:4 -server -rsh "ssh -x"
      Parsing application description...
      Identifying hosts...
      Spawning processes...
      Process Map:
        Process on host: fnx601
          rank 0
          rank 1
          rank 2
          rank 3

      Process Map End.
      Starting STAR-CCM+ parallel server
      Process layout for world 0 is as follows:
      mpirun:  proc 3259
        daemon proc 3262 on host 10.96.75.63
          rank 0:  proc 3887
          rank 1:  proc 3876
          rank 2:  proc 3893
          rank 3:  proc 3892
      libi40iw-i40iw_upost_send: post work request failed, invalid opcode: 0x3
      libi40iw-i40iw_upost_send: post work request failed, invalid opcode: 0x3
      libi40iw-i40iw_upost_send: post work request failed, invalid opcode: 0x3
      starccm+: Rank 0:1: MPI_Init: ibv_post_send() failed
      starccm+: Rank 0:0: MPI_Init: ibv_post_send() failed
      starccm+: Rank 0:1: MPI_Init: ibv_ring_sync() failed
      starccm+: Rank 0:0: MPI_Init: ibv_ring_sync() failed
      starccm+: Rank 0:1: MPI_Init: Internal Error: Processes cannot connect to rdma device
      starccm+: Rank 0:0: MPI_Init: Internal Error: Processes cannot connect to rdma device