Getting Started with Docker Container Over MLNX-OS

Version 9

    This post demonstrates how to get started and use docker over MLNX-OS.

    This feature is available starting with MLNX-OS version 3.6.4006





    Docker is a container that is invoked on top of MLNX-OS. Applications installed over docker can egress from the management port or the traffic ports (unlike VMs that can egress only from the management ports).





    1. Make sure you run MLNX-OS version 3.6.4006 or later.

    switch (config) # show version

    Product name:      MLNX-OS

    Product release:   3.6.4006

    Build ID:          #1-dev

    Build date:        2017-07-03 16:17:39

    Target arch:       x86_64

    Target hw:         x86_64

    Built by:          jenkins@a25f8aaaec03

    Version summary:   X86_64 3.6.4006 2017-07-03 16:17:39 x86_64


    3. Make sure that the switch clock is up-to-date (e.g. using NTP).

    switch (config) # show clock

    Time:       21:23:44

    Date:       2017/07/06

    Time zone:  UTC


    UTC offset: same as UTC


    switch (config) # show ntp

    NTP is administratively enabled.

    NTP Authentication is administratively disabled.

    Clock is synchronized.  Reference:  Offset: -3.611 ms.

    Active servers and peers:                                   

      Conf Type          : serv

      Status             : sys.peer(*)

      Stratum            : 3 

      Offset(msec)       : -3.611

      Ref clock          :

      Poll Interval (sec): 128

      Last Response (sec): 78 

      Auth state         : none   


    Note: Docker image pull command will not work if the switch's clock is not up-to-date.


    2. Enable docker on the switch.

    switch (config) # docker

    switch (config) # docker no shutdown


    3. Pull an image (e.g. CentOS, Ubuntu ...) from docker repository.

    switch  (config) # docker pull centos

    Using default tag: latest

    latest: Pulling from library/centos

    7b6bb4652a1b: Pull complete

    Digest: sha256:c1010e2fe2b635822d99a096b1f4184becf5d1c98707cbccae00be663a9b9131

    Status: Downloaded newer image for centos:latest


    switch  (config) # docker pull ubuntu

    Using default tag: latest

    latest: Pulling from library/ubuntu

    75c416ea735c: Pull complete

    Digest: sha256:a0ee7647e24c8494f1cf6b94f1a3cd127f423268293c25d924fbe18fd82db5a4

    Status: Downloaded newer image for ubuntu:latest


    4. Check images.

    switch (config) # show docker images


    Image              Version      Created            Size     


    centos             latest       24 hours ago       193MB

    ubuntu             latest       2 weeks ago        119MB


    5. Create a container from an image, select the persistence mode and start the container.


    A. To start a non-persistent container immediately, run:

    switch (config) # docker start centos latest my-container now

    Attempting to start docker container. Please wait (this can take a minute)...


    switch (config) # show docker ps


    Container           Image:Version           Created                Status                 


    my-container            centos:latest           About a minute ago     Up About a minute


    Note: This docker will not be invoked after the next reboot.


    B. To start a persistent container after reboot, run:

    switch (config) # docker start centos latest my-container init


    (config) # show running-config | include "docker"


       docker no shutdown

       docker start centos latest my-container init


    Note: This docker is not be invoked immediately, but just after the next reboot.



    Execute commands in the container.

    1. Enter to the docker bash for easy configuration.

    # docker exec my-container "bin/bash"

    Running exec_name:[bin/bash]



    Now you may begin typing Linux shell commands.


    2. For example, install net-tools.

    # yum install net-tools.x86_64



    Loaded plugins: fastestmirror, ovl

    Loading mirror speeds from cached hostfile

    * base:

    * extras:

    * updates:

    Resolving Dependencies

    --> Running transaction check

    ---> Package net-tools.x86_64 0:2.0-0.17.20131004git.el7 will be installed

    --> Finished Dependency Resolution


    Dependencies Resolved



    Package                                        Arch                                        Version                                                         Repository                                 Size



    net-tools                                      x86_64                                      2.0-0.17.20131004git.el7                                        base                                      304 k


    Transaction Summary


    Install  1 Package


    Total download size: 304 k

    Installed size: 917 k

    Is this ok [y/d/N]: y

    Downloading packages:

    warning: /var/cache/yum/x86_64/7/base/packages/net-tools-2.0-0.17.20131004git.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY

    Public key for net-tools-2.0-0.17.20131004git.el7.x86_64.rpm is not installed

    net-tools-2.0-0.17.20131004git.el7.x86_64.rpm                                                                                                                                        | 304 kB  00:00:00    

    Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

    Importing GPG key 0xF4A80EB5:

    Userid     : "CentOS-7 Key (CentOS 7 Official Signing Key) <>"

    Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5

    Package    : centos-release-7-3.1611.el7.centos.x86_64 (@CentOS)

    From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

    Is this ok [y/N]: y

    Running transaction check

    Running transaction test

    Transaction test succeeded

    Running transaction

      Installing : net-tools-2.0-0.17.20131004git.el7.x86_64                                                                                                                                                1/1

      Verifying  : net-tools-2.0-0.17.20131004git.el7.x86_64                                                                                                                                                1/1



      net-tools.x86_64 0:2.0-0.17.20131004git.el7                                                                                                                                                              




    2. Check the routing table to see that you have the routing capabilities via mgmt0 and the traffic ports.

    # route



    Kernel IP routing table

    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

    default         gateway         UG    1      0        0 mgmt0   UG    1      0        0 eth1.0   U     0      0        0 eth1.0   UG    1      0        0 eth1.0   U     0      0        0 swid0_eth.6     U     0      0        0 mgmt0     U     0      0        0 docker0


    Note: You see here the Linux routing table. Some of the entries get there via OSPF.

    This is the MLNX-OS routing table (type exit to exit the Linux shell).

    switch (config) # show ip route

    Flags: F - Failed to install in H/W

    VRF Name:         default          


       Destination     Mask            Gateway         Interface      Source AD/M      mgmt0          DHCP   0/0           mgmt0          direct 0/0           docker0        direct 0/0           eth1/1         ospf   110/2         eth1/1         direct 0/0           eth1/1         ospf   110/12         vlan6          direct 0/0   


    3. Check the docker interface (docker0).

    ifconfig docker0

    Running exec_name:[ifconfig docker0]


    docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

            inet  netmask  broadcast

            ether 02:42:b6:d4:f6:9b  txqueuelen 0  (Ethernet)

            RX packets 0  bytes 0 (0.0 B)

            RX errors 0  dropped 0  overruns 0  frame 0

            TX packets 0  bytes 0 (0.0 B)

            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


    4. Ping a server inband, via the data ports (e.g. 1.1.1.x network).

    switch (config) # ping                                  < -- via the MLNX-OS

    PING ( 56(84) bytes of data.

    64 bytes from icmp_seq=1 ttl=63 time=0.233 ms

    64 bytes from icmp_seq=2 ttl=63 time=0.170 ms


    switch (config) # docker exec my-container "ping"      < -- via the docker interface

    Running exec_name:[ping]


    PING ( 56(84) bytes of data.

    64 bytes from icmp_seq=1 ttl=63 time=0.202 ms

    64 bytes from icmp_seq=2 ttl=63 time=0.177 ms


    # ping                                                                           < -- via docker shell

    PING ( 56(84) bytes of data.

    64 bytes from icmp_seq=1 ttl=63 time=0.166 ms

    64 bytes from icmp_seq=2 ttl=63 time=0.179 ms


    For all docker commands, see the MLNX-OS User Manual.


    To see an application example over docker, refer to  HowTo enable Wireshark over Docker Container (MLNX-OS).



    1. If you receive the following message when trying to pull a new image:

    switch  (config) # docker pull centos

    Using default tag: latest

    Error response from daemon: Get x509: certificate has expired or is not yet valid


    Make sure that the clock is up-to-date. See HowTo Enable NTP on Mellanox Switches to set it up.


    2. If the commands ifconfig/ip link don't work, you need to install the net-tools package. See examples above.