12 Replies Latest reply on Jan 13, 2014 6:56 AM by rian

    MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

      Hi

       

      Just try to install MLNX_OFED_LINUX-2.0-3.0.0-rhel6.4-x86_64.iso on Oracle Enterprise Linux 6.4. The command  ./mlnxofedinstall gives error: The current MLNX_OFED_LINUX is intended for rhel6.4

      uname -r is 2.6.32-358.el6.x86_64 (change to rehl kernel)

      Tried also to run new installation package:

      ./mlnx_add_kernel_support.sh -m /mnt --make-tgz -k 2.6.32-358.el6.x86_64 -s /lib/modules/2.6.32-358.el6.x86_64/build -n MLNX-NEW

      creation success, but when I try to install the same error:

      error: The current MLNX_OFED_LINUX is intended for rhel6.4

       

      Any Idea ?

       

      Many thanks

        • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4
          siszlai

          You can skip the distro check with --skip-distro-check

          You might also need --without-depcheck

           

          Cheers,

          sorin

          • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

            Hi,

            Since you're using OEL 6.4 ,I would recommend downloading the version for OEL 6.4

             

            Here the Iso link:

            Mellanox Technologies: MLNX_OFED EULA

             

            Or TGZ file"

            MLNX_OFED_LINUX-2.0-3.0.0-oel6.4-x86_64.tgz

            • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

              Hi

               

              Thanks that helped out with --skip-distro-check

               

              My question now, I think hardware is working. But I dont see any Ib0 interface ? How to get the infiniband interface name ?

               

              Thanks

               

              # ./hca_self_test.ofed

               

              ---- Performing Adapter Device Self Test ----

              Number of CAs Detected ................. 1

              PCI Device Check ....................... PASS

              Kernel Arch ............................ x86_64

              Host Driver Version .................... MLNX_OFED_LINUX-2.0-3.0.0 (OFED-2.0-3.0.0): 2.6.32-358.el6.x86_64

              Host Driver RPM Check .................. PASS

              Firmware on CA #0 VPI .................. v2.30.3200

              Firmware Check on CA #0 (VPI) .......... PASS

                  NOTE: The found fw version is higher than the fw included in this package (v2.30.3110)

              Host Driver Initialization ............. PASS

              Number of CA Ports Active .............. 0

              Kernel Syslog Check .................... PASS

              Node GUID on CA #0 (VPI) ............... NA

              ------------------ DONE ---------------------

              • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

                Ok did install OEL OFED.

                Still have the issue that I dont see the ib0

                Could it be that I have already configured eth1 as ethernt ?

                see output (is empty):

                 

                #ibv_devices

                    device                 node GUID

                    ------              ----------------

                #

                • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

                  did execute the commands, please see below:

                   

                  [root@xxx]# lspci -d 15b3:

                  03:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

                   

                  [root@xxx]# lsmod | grep mlx

                  mlx5_ib                78913  0

                  mlx5_core              72388  1 mlx5_ib

                  mlx4_en                78738  0

                  mlx4_ib               155891  0

                  ib_sa                  24010  5 rdma_ucm,rdma_cm,ib_ipoib,ib_cm,mlx4_ib

                  ib_mad                 43177  4 ib_cm,ib_umad,mlx4_ib,ib_sa

                  ib_core                81943  12 rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_uverbs,ib_umad,mlx5_ib,mlx4_ib,ib_sa,ib_mad

                  mlx4_core             240441  2 mlx4_en,mlx4_ib

                  compat                 22986  17 rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_addr,ib_ipoib,ib_cm,ib_uverbs,ib_umad,mlx5_ib,mlx5_core,mlx4_en,mlx4_ib,ib_sa,ib_mad,ib_core,mlx4_core

                   

                  [root@xxx]# ibv_devices

                      device                 node GUID

                      ------              ----------------

                   

                  .....still no device visible, any other idea ? Many thanks in advance

                  • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4
                    siszlai

                    Can you check this dir?

                     

                    # ls -l /dev/infiniband/

                    total 0

                    crw-rw---- 1 root root 231,  64 Jan  6 18:57 issm0

                    crw-rw---- 1 root root 231,  65 Jan  6 18:57 issm1

                    crw-rw-rw- 1 root root  10,  56 Dec 28 00:17 rdma_cm

                    crw-rw-rw- 1 root root 231, 224 Dec 28 00:17 ucm0

                    crw-rw-rw- 1 root root 231,   0 Jan  6 18:57 umad0

                    crw-rw-rw- 1 root root 231,   1 Jan  6 18:57 umad1

                    crw-rw-rw- 1 root root 231, 192 Dec 28 00:17 uverbs0

                     

                    Those are created by udev and the rules should look like this.

                    If those rules are missing or have wrong permissions, that might be the cause.

                     

                    # cat /etc/udev/rules.d/90-ib.rules

                    KERNEL=="umad*", NAME="infiniband/%k" MODE="0666"

                    KERNEL=="issm*", NAME="infiniband/%k"

                    KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666"

                    KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666"

                    KERNEL=="ucma", NAME="infiniband/%k", MODE="0666"

                    KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666"

                    • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

                      Hi here is the output:

                       

                      # ls -l /dev/infiniband/
                      total 0
                      crw-rw-rw-. 1 root root 10, 56 Jan  1 10:33 rdma_cm

                       

                      and this:

                       

                      # cat /etc/udev/rules.d/90-ib.rules
                      KERNEL=="umad*", NAME="infiniband/%k" MODE="0666"
                      KERNEL=="issm*", NAME="infiniband/%k"
                      KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666"
                      KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666"
                      KERNEL=="ucma", NAME="infiniband/%k", MODE="0666"
                      KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666"

                       

                       

                      What to do next ?

                        • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4
                          siszlai

                          Stop openibd

                          Turn debug on for udev.

                           

                          udevadm control --log-priority debug

                          udevadm control --reload-rules

                           

                          Start openibd

                           

                          And search for 90-ib.rules in /var/log/messages

                          These are logs when it works, but you should see errors that can give use some clues

                           

                          Jan  6 23:08:40 my-host udevd-work[11143]: MODE 0666 /etc/udev/rules.d/90-ib.rules:3

                          Jan  6 23:08:40 my-host udevd-work[11143]: NAME 'infiniband/ucm0' /etc/udev/rules.d/90-ib.rules:3

                          Jan  6 23:08:40 my-host udevd-work[11143]: create db link (infiniband/ucm0 char/231:224)

                          Jan  6 23:08:40 my-host udevd-work[11143]: creating device node '/dev/infiniband/ucm0', devnum=231:224, mode=0666, uid=0, gid=0

                          Jan  6 23:08:40 my-host udevd-work[11143]: preserve file '/dev/infiniband/ucm0', because it has correct dev_t

                          Jan  6 23:08:40 my-host udevd-work[11143]: preserve permissions /dev/infiniband/ucm0, 020666, uid=0, gid=0

                          Jan  6 23:08:40 my-host udevd-work[11143]: creating symlink '/dev/char/231:224' to '../infiniband/ucm0'

                        • Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

                          I did plug in the 2 infiniband cables and checked if they are correct plugged in into port 1 and 2.

                          Now output looks like this:

                           

                          [root@xxx ~]# ibv_devices

                              device                 node GUID

                              ------              ----------------

                              mlx4_0              0002c90300345f50

                          [root@xxx ~]# lspci -d 15b3:

                          03:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

                          [root@xxx ~]# ls -l /dev/infiniband/

                          total 0

                          crw-rw----. 1 root root 231,  64 Jan  7 10:10 issm0

                          crw-rw----. 1 root root 231,  65 Jan  7 10:10 issm1

                          crw-rw-rw-. 1 root root  10,  56 Jan  7 10:10 rdma_cm

                          crw-rw-rw-. 1 root root 231, 224 Jan  7 10:10 ucm0

                          crw-rw-rw-. 1 root root 231,   0 Jan  7 10:10 umad0

                          crw-rw-rw-. 1 root root 231,   1 Jan  7 10:10 umad1

                          crw-rw-rw-. 1 root root 231, 192 Jan  7 10:10 uverbs0

                          [root@xxx ~]# ls /sys/class/infiniband

                          mlx4_0

                          [root@ewsexaobi01 ~]# ibv_devinfo

                          hca_id: mlx4_0

                                  transport:                      InfiniBand (0)

                                  fw_ver:                         2.30.3200

                                  node_guid:                      0002:c903:0034:5f50

                                  sys_image_guid:                 0002:c903:0034:5f53

                                  vendor_id:                      0x02c9

                                  vendor_part_id:                 4099

                                  hw_ver:                         0x1

                                  board_id:                       HP_0230240019

                                  phys_port_cnt:                  2

                                          port:   1

                                                  state:                  PORT_ACTIVE (4)

                                                  max_mtu:                4096 (5)

                                                  active_mtu:             4096 (5)

                                                  sm_lid:                 2

                                                  port_lid:               13

                                                  port_lmc:               0x00

                                                  link_layer:             InfiniBand

                           

                                          port:   2

                                                  state:                  PORT_ACTIVE (4)

                                                  max_mtu:                4096 (5)

                                                  active_mtu:             4096 (5)

                                                  sm_lid:                 2

                                                  port_lid:               14

                                                  port_lmc:               0x00

                                                  link_layer:             InfiniBand

                           

                          Is this ok now ? How to get now network configured ip ?