1 Reply Latest reply on Mar 15, 2018 11:24 AM by doleary

    Connect-X3 how to create VFs on multiple cards

    doleary

      I've been struggling to get VFs working with two Connect-X3 card in the same chassis.  The chassis is a NUMA system with one card associated with each processor.  The 4 PFs are showing up in Linux, but I need to have 3 VFs per physical PF, Can somebody please advise on how to configure the Mellanox driver for this?

       

      I can get the driver to correctly create the VFs for the first card alright, but I can't get it to create any VFs at all on the 2nd card

      Here's the contents of /etc/modprobe.d/mlx4.conf with the variations I've tried in comments:

       

      lab@rpdsim:/etc/modprobe.d$ cat mlx4.conf

      # mlx4_core gets automatically loaded, load mlx4_en also (LP: #1115710)

      softdep mlx4_core post: mlx4_en

       

      #This line kind of works, but only creates VFs for the 0000:82:00.0 card

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=3,3,0 probe_vf=3,3,0

       

      #This line doesn't work at all

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:06:00.0,0000:82:00.0;3;3 probe_vf=0000:06:00.0,0000:82:00.0;3;3

       

      #This line doesn't work at all

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:82:00.0;3 probe_vf=0000:82:00.0;3

       

      #This line doesn't work at all

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:82:00.3-5;3 probe_vf=0000:82:00.3-5;3

       

      #This line doesn't work at all

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:82:00.3-5;3;3;0 probe_vf=0000:82:00.3-5;3;3;0

       

      #This line kind of works, but only creates VFs for the 0000:82:00.0 card

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:82:00.0-3 probe_vf=0000:82:00.0-3

       

      #This line kind of works, but only creates VFs for the 0000:82:00.0 card

      #options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:06:00.0-3,0000:82:00.0-3 probe_vf=0000:06:00.0-3,0000:82:00.0-3

       

      #This line doesn't work at all

      options mlx4_core log_num_mgm_entry_size=-7 enable_sys_tune=1 mlx4_en_only_mode=1 num_vfs=0000:06:00.0-3 probe_vf=0000:06:00.0-3

       

      Thanks

         Donal