Getting Started with Docker Container Over MLNX-OS

Version 9

    This post demonstrates how to get started and use docker over MLNX-OS.

    This feature is available starting with MLNX-OS version 3.6.4006

     

    References

     

    Overview

    Docker is a container that is invoked on top of MLNX-OS. Applications installed over docker can egress from the management port or the traffic ports (unlike VMs that can egress only from the management ports).

     

     

     

    Configuration

    1. Make sure you run MLNX-OS version 3.6.4006 or later.

    switch (config) # show version

    Product name:      MLNX-OS

    Product release:   3.6.4006

    Build ID:          #1-dev

    Build date:        2017-07-03 16:17:39

    Target arch:       x86_64

    Target hw:         x86_64

    Built by:          jenkins@a25f8aaaec03

    Version summary:   X86_64 3.6.4006 2017-07-03 16:17:39 x86_64

     

    3. Make sure that the switch clock is up-to-date (e.g. using NTP).

    switch (config) # show clock

    Time:       21:23:44

    Date:       2017/07/06

    Time zone:  UTC

                (Etc/UTC)

    UTC offset: same as UTC

     

    switch (config) # show ntp

    NTP is administratively enabled.

    NTP Authentication is administratively disabled.

    Clock is synchronized.  Reference: 10.4.0.134.  Offset: -3.611 ms.

    Active servers and peers:

    10.4.0.134                                   

      Conf Type          : serv

      Status             : sys.peer(*)

      Stratum            : 3 

      Offset(msec)       : -3.611

      Ref clock          : 192.114.62.250

      Poll Interval (sec): 128

      Last Response (sec): 78 

      Auth state         : none   

     

    Note: Docker image pull command will not work if the switch's clock is not up-to-date.

     

    2. Enable docker on the switch.

    switch (config) # docker

    switch (config) # docker no shutdown

     

    3. Pull an image (e.g. CentOS, Ubuntu ...) from docker repository.

    switch  (config) # docker pull centos

    Using default tag: latest

    latest: Pulling from library/centos

    7b6bb4652a1b: Pull complete

    Digest: sha256:c1010e2fe2b635822d99a096b1f4184becf5d1c98707cbccae00be663a9b9131

    Status: Downloaded newer image for centos:latest

     

    switch  (config) # docker pull ubuntu

    Using default tag: latest

    latest: Pulling from library/ubuntu

    75c416ea735c: Pull complete

    Digest: sha256:a0ee7647e24c8494f1cf6b94f1a3cd127f423268293c25d924fbe18fd82db5a4

    Status: Downloaded newer image for ubuntu:latest

     

    4. Check images.

    switch (config) # show docker images

    -------------------------------------------------------------

    Image              Version      Created            Size     

    -------------------------------------------------------------

    centos             latest       24 hours ago       193MB

    ubuntu             latest       2 weeks ago        119MB

     

    5. Create a container from an image, select the persistence mode and start the container.

     

    A. To start a non-persistent container immediately, run:

    switch (config) # docker start centos latest my-container now

    Attempting to start docker container. Please wait (this can take a minute)...

     

    switch (config) # show docker ps

    -------------------------------------------------------------------------------------------

    Container           Image:Version           Created                Status                 

    -------------------------------------------------------------------------------------------

    my-container            centos:latest           About a minute ago     Up About a minute

     

    Note: This docker will not be invoked after the next reboot.

     

    B. To start a persistent container after reboot, run:

    switch (config) # docker start centos latest my-container init

     

    (config) # show running-config | include "docker"

       docker

       docker no shutdown

       docker start centos latest my-container init

     

    Note: This docker is not be invoked immediately, but just after the next reboot.

     

    Usage

    Execute commands in the container.

    1. Enter to the docker bash for easy configuration.

    # docker exec my-container "bin/bash"

    Running exec_name:[bin/bash]

    #

     

    Now you may begin typing Linux shell commands.

     

    2. For example, install net-tools.

    # yum install net-tools.x86_64

     

     

    Loaded plugins: fastestmirror, ovl

    Loading mirror speeds from cached hostfile

    * base: mirror.pac-12.org

    * extras: mirrors.sonic.net

    * updates: mirrors.kernel.org

    Resolving Dependencies

    --> Running transaction check

    ---> Package net-tools.x86_64 0:2.0-0.17.20131004git.el7 will be installed

    --> Finished Dependency Resolution

     

    Dependencies Resolved

     

    ============================================================================================================================================================================================================

    Package                                        Arch                                        Version                                                         Repository                                 Size

    ============================================================================================================================================================================================================

    Installing:

    net-tools                                      x86_64                                      2.0-0.17.20131004git.el7                                        base                                      304 k

     

    Transaction Summary

    ============================================================================================================================================================================================================

    Install  1 Package

     

    Total download size: 304 k

    Installed size: 917 k

    Is this ok [y/d/N]: y

    Downloading packages:

    warning: /var/cache/yum/x86_64/7/base/packages/net-tools-2.0-0.17.20131004git.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY

    Public key for net-tools-2.0-0.17.20131004git.el7.x86_64.rpm is not installed

    net-tools-2.0-0.17.20131004git.el7.x86_64.rpm                                                                                                                                        | 304 kB  00:00:00    

    Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

    Importing GPG key 0xF4A80EB5:

    Userid     : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"

    Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5

    Package    : centos-release-7-3.1611.el7.centos.x86_64 (@CentOS)

    From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

    Is this ok [y/N]: y

    Running transaction check

    Running transaction test

    Transaction test succeeded

    Running transaction

      Installing : net-tools-2.0-0.17.20131004git.el7.x86_64                                                                                                                                                1/1

      Verifying  : net-tools-2.0-0.17.20131004git.el7.x86_64                                                                                                                                                1/1

     

    Installed:

      net-tools.x86_64 0:2.0-0.17.20131004git.el7                                                                                                                                                              

     

    Complete!

     

    2. Check the routing table to see that you have the routing capabilities via mgmt0 and the traffic ports.

    # route

     

     

    Kernel IP routing table

    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

    default         gateway         0.0.0.0         UG    1      0        0 mgmt0

    1.1.1.0         1.1.2.1         255.255.255.0   UG    1      0        0 eth1.0

    1.1.2.0         0.0.0.0         255.255.255.0   U     0      0        0 eth1.0

    1.1.5.0         1.1.2.1         255.255.255.0   UG    1      0        0 eth1.0

    1.1.6.0         0.0.0.0         255.255.255.0   U     0      0        0 swid0_eth.6

    10.20.0.0       0.0.0.0         255.255.0.0     U     0      0        0 mgmt0

    172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0

     

    Note: You see here the Linux routing table. Some of the entries get there via OSPF.

    This is the MLNX-OS routing table (type exit to exit the Linux shell).

    switch (config) # show ip route

    Flags: F - Failed to install in H/W

    VRF Name:         default          

    -----------------------------

       Destination     Mask            Gateway         Interface      Source AD/M  

       0.0.0.0         0.0.0.0         10.20.0.251     mgmt0          DHCP   0/0   

       10.20.0.0       255.255.0.0     0.0.0.0         mgmt0          direct 0/0   

       172.17.0.0      255.255.0.0     0.0.0.0         docker0        direct 0/0   

       1.1.1.0         255.255.255.0   1.1.2.1         eth1/1         ospf   110/2 

       1.1.2.0         255.255.255.0   0.0.0.0         eth1/1         direct 0/0   

       1.1.5.0         255.255.255.0   1.1.2.1         eth1/1         ospf   110/12

       1.1.6.0         255.255.255.0   0.0.0.0         vlan6          direct 0/0   

     

    3. Check the docker interface (docker0).

    ifconfig docker0

    Running exec_name:[ifconfig docker0]

     

    docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

            inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0

            ether 02:42:b6:d4:f6:9b  txqueuelen 0  (Ethernet)

            RX packets 0  bytes 0 (0.0 B)

            RX errors 0  dropped 0  overruns 0  frame 0

            TX packets 0  bytes 0 (0.0 B)

            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

     

    4. Ping a server inband, via the data ports (e.g. 1.1.1.x network).

    switch (config) # ping 1.1.1.2                                  < -- via the MLNX-OS

    PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.

    64 bytes from 1.1.1.2: icmp_seq=1 ttl=63 time=0.233 ms

    64 bytes from 1.1.1.2: icmp_seq=2 ttl=63 time=0.170 ms

     

    switch (config) # docker exec my-container "ping 1.1.1.2"      < -- via the docker interface

    Running exec_name:[ping 1.1.1.2]

     

    PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.

    64 bytes from 1.1.1.2: icmp_seq=1 ttl=63 time=0.202 ms

    64 bytes from 1.1.1.2: icmp_seq=2 ttl=63 time=0.177 ms

     

    # ping 1.1.1.2                                                                           < -- via docker shell

    PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.

    64 bytes from 1.1.1.2: icmp_seq=1 ttl=63 time=0.166 ms

    64 bytes from 1.1.1.2: icmp_seq=2 ttl=63 time=0.179 ms

     

    For all docker commands, see the MLNX-OS User Manual.

     

    To see an application example over docker, refer to  HowTo enable Wireshark over Docker Container (MLNX-OS).

     

    Troubleshooting

    1. If you receive the following message when trying to pull a new image:

    switch  (config) # docker pull centos

    Using default tag: latest

    Error response from daemon: Get https://registry-1.docker.io/v2/: x509: certificate has expired or is not yet valid

     

    Make sure that the clock is up-to-date. See HowTo Enable NTP on Mellanox Switches to set it up.

     

    2. If the commands ifconfig/ip link don't work, you need to install the net-tools package. See examples above.