HowTo Test 1PPS on Mellanox Adapters

Version 2

    One Pulse Per Second (1PPS) is a time synchronization feature that allows the adapter to be able to send or receive 1 pulse per second on a dedicated pin on the adapter card.

    The supported ASICs are ConnectX-4 or later, using MLNX_OFED 4.1 or later. Only one pin is supported and could be configured as 1PPS in or 1PPS out.

    The adapter to be used with this feature, has an SMA connector and has the P/N of MCX556A-EDAM (FSI card).

    testptp.c is a short script that can help you test time syncronization features such as One Pulse Per Second (1PPS).

     

    References

     

    Setup

    In this example we have two servers connected back to back with the 1PPS pin on the adapter. One server will generate the 1PPS signal (1PPS out), while the other server will be receiving the 1PPS (1PPS in).

     

    Installation

    1. There are several location to download the script, make sure to download one of the latest ones. here is an example.

    https://www.mjmwired.net/kernel/Documentation/ptp/testptp.c

     

    2. Install the script using:

     

    # gcc -Wall -lrt testptp.c -o testptp

     

     

    3. Get help

    # ./testptp -h

    usage: testptp [options]

    -a val     request a one-shot alarm after 'val' seconds

    -A val     request a periodic alarm every 'val' seconds

    -c         query the ptp clock's capabilities

    -d name    device to open

    -e val     read 'val' external time stamp events

    -f val     adjust the ptp clock frequency by 'val' ppb

    -g         get the ptp clock time

    -h         prints this message

    -i val     index for event/trigger

    -k val     measure the time offset between system and phc clock

                for 'val' times (Maximum 25)

    -l         list the current pin configuration

    -L pin,val configure pin index 'pin' with function 'val'

                the channel index is taken from the '-i' option

                'val' specifies the auxiliary function:

                0 - none

                1 - external time stamp

                2 - periodic output

    -p val     enable output with a period of 'val' nanoseconds

    -P val     enable or disable (val=1|0) the system clock PPS

    -s         set the ptp clock time from the system time

    -S         set the system time from the ptp clock time

    -t val     shift the ptp clock time by 'val' seconds

    -T val     set the ptp clock time to 'val' seconds

     

    Configuration

    1. Make sure to install MLNX_OFED 4.1 or above, on your server.

     

    2. Get the Time Synchronization support on the relevant port via ethtool. In this example, we are using PTP Clock 3. The device path will be /dev/ptp3

    # ethtool -T ens2

    Time stamping parameters for ens2:

    Capabilities:

      hardware-transmit     (SOF_TIMESTAMPING_TX_HARDWARE)

      software-transmit     (SOF_TIMESTAMPING_TX_SOFTWARE)

      hardware-receive      (SOF_TIMESTAMPING_RX_HARDWARE)

      software-receive      (SOF_TIMESTAMPING_RX_SOFTWARE)

      software-system-clock (SOF_TIMESTAMPING_SOFTWARE)

      hardware-raw-clock    (SOF_TIMESTAMPING_RAW_HARDWARE)

    PTP Hardware Clock: 3

    Hardware Transmit Timestamp Modes:

      on                    (HWTSTAMP_TX_ON)

      one-step-sync         (HWTSTAMP_TX_ONESTEP_SYNC)

    Hardware Receive Filter Modes:

      all                   (HWTSTAMP_FILTER_ALL)

      some                  (HWTSTAMP_FILTER_SOME)

     

    3. Query the ptp clock's capabilities.

    # ./testptp -d /dev/ptp3 -c

    capabilities:

      100000000 maximum frequency adjustment (ppb)

      0 programmable alarms

      1 external time stamp channels

      1 programmable periodic signals

      1 pulse per second

      1 programmable pins

     

    4. Check the current pin configuration

    # ./testptp -d /dev/ptp3 -l

    name mlx5_pps0 index 0 func 0 chan 0

    • Name: Pin name.
    • index: The Pin number. In case of ConnectX-4, we support only one Pin, therefore it will always be zero.
    • func (function): The pin Configuration

                0 - none

                1 - 1PPS In

                2 - 1PPS Out (periodic output)

    • chan (channel) - Reserved field.

     

    The parameters are taken from ptp_pin_desc in ptp_clock.h .

     

    5. On one server, set 1PPS in on the adapter, use pin index 0 and -L flag and value 1. Check the func value was changed.

    # ./testptp -d /dev/ptp3 -l

    name mlx5_pps0 index 0 func 0 chan 0

     

    # ./testptp -d /dev/ptp3 -L 0,1

    set pin function okay

     

    # ./testptp -d /dev/ptp3 -l

    name mlx5_pps0 index 0 func 1 chan 0

     

    6. On the second server, set 1PPS out on the adapter, use pin index 0 the -L flag and value 2. Check the func value was changed.

    # ./testptp -d /dev/ptp3 -l
    name mlx5_pps0 index 0 func 0 chan 0

     

    # ./testptp -d /dev/ptp3 -L 0,2

    set pin function okay

    # ./testptp -d /dev/ptp3 -l

    name mlx5_pps0 index 0 func 2 chan 0

     

    7. On the 1PPS in server, set the maximum number of events to be set/handled by the application.

    #./testptp -d /dev/ptp3 -e 1000

    external time stamp request okay

    ...

    8. On the 1PPS out server, enabled 1PPS using -p 1000000000 (in nsec)

     

    # ./testptp -d /dev/ptp3 -p 1000000000
    periodic output request okay

     

     

    9. Check the 1PPS in server to get the time stamp.

    # ./testptp -d /dev/ptp3 -e 1000

    external time stamp request okay

    event index 0 at 1496670401.396081197

    event index 0 at 1496670402.396060918

    event index 0 at 1496670403.396040626

    event index 0 at 1496670404.396020335

    event index 0 at 1496670405.396000037

    event index 0 at 1496670406.395979745

    event index 0 at 1496670407.395959460

    event index 0 at 1496670408.395939174

    event index 0 at 1496670409.395918883

    event index 0 at 1496670410.395898597

     

    10. Some applications may need to read the timestamp from a file. To write the time stamps into a file, enable the pps_enable sysfs parameter in the 1 PPS in server

    # echo 1 > /sys/class/ptp/ptp3/pps_enable

    # watch -n 1 'cat /sys/class/pps/pps0/assert'

     

    No changes are needed on the other server.

     

    To disable it again and get the timestamps to the terminal, run:

    # echo 0 > /sys/class/ptp/ptp3/pps_enable

    # ./testptp -d /dev/ptp3 -e 1000

     

    Command Summary

     

    1PPS in

    ethtool -T ens2

    ./testptp -d /dev/ptp3 -c

    ./testptp -d /dev/ptp3 -l

    ./testptp -d /dev/ptp3 -L 0,0

    ./testptp -d /dev/ptp3 -l

    ./testptp -d /dev/ptp3 -L 0,1

    ./testptp -d /dev/ptp3 -e 1000

    echo 1 > /sys/class/ptp/ptp3/pps_enable

    watch -n 1 'cat /sys/class/pps/pps0/assert'

    echo 0 > /sys/class/ptp/ptp3/pps_enable

    ./testptp -d /dev/ptp3 -e 1000

     

    1PPS out

    ethtool -T ens2

    ./testptp -d /dev/ptp3 -c

    ./testptp -d /dev/ptp3 -l

    ./testptp -d /dev/ptp3 -L 0,0

    ./testptp -d /dev/ptp3 -l

    ./testptp --help

    ./testptp -d /dev/ptp3 -L 0,2

    ./testptp -d /dev/ptp3 -p 1000000000