17 Replies Latest reply on May 6, 2016 9:05 AM by praetzel

    установка OFED на Ubuntu 14.10 (проблемы с прошивкой сетевого адаптера)

      Доброго времени суток! Возникла проблема с установкой библиотеки OFED на Ubuntu 14.10.

       

      На сервере установлена сетевая карточка  Mellanox Technologies MT25204 [InfiniHost III Lx HCA].

       

      Скачал пакет MLNX_OFED_LINUX-2.4-1.0.4-ubuntu14.10-x86_64.tgz. Распаковал, установил все зависимости и запустил установку.

       

      Установка прервалась на следующей операции: "Querying Mellanox devices firmware ..."

       

      В консоль вывелось следующее сообщение об ошибке: "Failed to query 0000: 07 : 00.0 device, error: File exists. MFE_OLD_DEVICE_TYPE"

       

      Не могу понять, почему устройство не ответило на запрос. Буду рад любым советам.

        • Re: установка OFED на Ubuntu 14.10 (проблемы с прошивкой сетевого адаптера)
          alkx

          Here are the steps that might solve the issue

          Before installing MOFED

          Download Mellanox Firmware tool

          Run

          mst start

          mst status

          flind -d /dev/mst/PATH_TO_DEVICE q

          Obtain PSID

          Find the latest firmware on mellanox site coresponing your to PSID

          Burn the firmware like this

          flint -d /dev/mst/PATH -i IMAGE_ANEM --nofs burn

           

          This HCA probably is old type card that their flash size is smaller

            • Re: установка OFED на Ubuntu 14.10 (проблемы с прошивкой сетевого адаптера)

              Я выполнил все указанные действия:

               

              root@ivan-X7DWT:~#

              root@ivan-X7DWT:~# mst start

              Starting MST (Mellanox Software Tools) driver set

              Loading MST PCI module - Success

              Loading MST PCI configuration module - Success

              Create devices

              root@ivan-X7DWT:~# mst status

              MST modules:

              ------------

                  MST PCI module loaded

                  MST PCI configuration module loaded

               

               

              MST devices:

              ------------

              /dev/mst/mt25204_pciconf0        - PCI configuration cycles access.

                                                 domain:bus:dev.fn=0000:07:00.0 addr.reg=88 data.reg=92

                                                 Chip revision is: A0

              /dev/mst/mt25204_pci_cr0         - PCI direct access.

                                                 domain:bus:dev.fn=0000:07:00.0 bar=0xd8800000 size=0x100000

                                                 Chip revision is: A0

              root@ivan-X7DWT:~# flint -d /dev/mst/mt25204_pciconf0 q

              -E- Cannot open Device: /dev/mst/mt25204_pciconf0. No such file or directory MFE_OLD_DEVICE_TYPE

              root@ivan-X7DWT:~# flint -d /dev/mst/mt25204_pci_cr0 q

              -E- Cannot open Device: /dev/mst/mt25204_pci_cr0. No such file or directory MFE_OLD_DEVICE_TYPE

              root@ivan-X7DWT:~#

               

              не понимаю в чем дело...

            • Re: ????????? OFED ?? Ubuntu 14.10 (???????? ? ????????? ???????? ????????)
              praetzel

              I've run into the exact same problem.

              I started with fully patched CentOS 7 on Asus and SuperMicro motherboards using the stock InfiBand software.  mst start would not create devices on CentOS 7 - although lsci showed them to be there and it looked like all appropiate drivers were installed

              The InfiniHost III boards (I've tested 4) are taken from Isilon IQ6000 and IQ12000 systems.

              A friend experienced with IB hardware said that the firmware should be reflashed but I've not managed to do that.  He's been unable to get the cards to work in a server which also has newer Mellanox cards.

               


                [root ~]# lspci | grep Mellanox
                02:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20)
               
                [root ~]# mst start
                Starting MST (Mellanox Software Tools) driver set
                Loading MST PCI module - Success
                Loading MST PCI configuration module - Success
                Create devices
               
                [root ~]# mst status
                MST modules:
                ------------
                    MST PCI module loaded
                    MST PCI configuration module loaded
               
                MST devices:
                ------------
                /dev/mst/mt25208_pciconf0        - PCI configuration cycles access.
                                                   bus:dev.fn=02:00.0 addr.reg=88 data.reg=92
                                                   Chip revision is: A0
                /dev/mst/mt25208_pci_cr0         - PCI direct access.
                                                   bus:dev.fn=02:00.0 bar=0xfea00000 size=0x100000
                                                   Chip revision is: A0
                /dev/mst/mt25208_pci_ddr0        - PCI direct access.
                                                   bus:dev.fn=02:00.0 bar=0xdf800000 size=0x800000
               
               
                [root ~]# ibv_devinfo
                No IB devices found
               
               
                [root ~]# flint -d /dev/mst/mt25208_pci_cr0 query
                 Received CFI query from addr 0x10: [▒&]
                 Received CFI query from addr 0x20: [Z]
                -E- Can not open /dev/mst/mt25208_pci_cr0:  MFE_CFI_FAILED
               
               
                [root MLNX_OFED_LINUX-1.5.3-4.0.42-rhel5.10-x86_64]# mstflint -d 0:02.0 -i fw-25208-4_8_200-MHEL-CF128.bin b
                Warning: memory access to device 0:02.0 failed: No such device or address.
                Warning: Fallback on IO: much slower, and unsafe if device in use.
                -E- Can not open 0:02.0: No such file or directory MFE_CR_ERROR
               
                [root MLNX_OFED_LINUX-1.5.3-4.0.42-rhel5.10-x86_64]# mstflint -d /dev/mst/mt25209_pciconf0 -i fw-25208-4_8_200-MHEL-CF128.bin b
                Unable to parse device name /dev/mst/mt25209_pciconf0
                -E- Can not open /dev/mst/mt25209_pciconf0: Invalid argument MFE_CR_ERROR
               
               
                [root ~]# lspci |grep Mellanox | awk '{print $1}' | xargs -i -r lspci -s {} -xxxvvv
                02:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20)
                        Subsystem: Device 4953:0008
                        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
                        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                        Latency: 0, Cache Line Size: 64 bytes
                        Interrupt: pin A routed to IRQ 169
                        Region 0: Memory at fea00000 (64-bit, non-prefetchable) [size=1M]
                        Region 2: Memory at df800000 (64-bit, prefetchable) [size=8M]
                        Region 4: Memory at a0000000 (64-bit, prefetchable) [size=512M]
                        Capabilities: [40] Power Management version 2
                                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
                        Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
                                Address: 0000000000000000  Data: 0000
                        Capabilities: [84] MSI-X: Enable- Count=32 Masked-
                                Vector table: BAR=0 offset=00082000
                                PBA: BAR=0 offset=00082200
                        Capabilities: [60] Express (v1) Endpoint, MSI 00
                                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
                                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                                        MaxPayload 128 bytes, MaxReadReq 4096 bytes
                                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                                LnkCap: Port #8, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 unlimited
                                        ClockPM- Surprise- LLActRep- BwNot-
                                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                                LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                        Kernel modules: ib_mthca
                00: b3 15 78 62 06 01 10 00 20 00 06 0c 10 00 00 00
                10: 04 00 a0 fe 00 00 00 00 0c 00 80 df 00 00 00 00
                20: 0c 00 00 a0 00 00 00 00 00 00 00 00 53 49 08 00
                30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 00 00
                40: 01 90 02 00 00 00 00 00 03 90 ff 7f 11 11 00 00
                50: 00 00 00 00 00 00 00 00 14 00 0f 00 78 62 a0 02
                60: 10 00 01 00 20 0e 2c 01 00 50 00 00 81 f4 03 08
                70: 40 00 81 10 00 00 00 00 00 00 00 00 00 00 00 00
                80: 00 00 00 00 11 60 1f 00 00 20 08 00 00 22 08 00
                90: 05 84 8a 00 00 00 00 00 00 00 00 00 00 00 00 00
                a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
                f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00