HowTo Install Flexible I/O (Fio) for Storage Benchmarking I/O Testing

Version 4

    This post discusses the download, compilation, and use of Flexible I/O (fio) package for I/O benchmarking.





    1. Download the latest fio package.

    # wget .


    2. Unzip, and open the package.

    # gunzip fio-2.1.10.tar.gz

    # tar -xf fio-2.1.10.tar


    3. Configure, run make and make install.

    # cd fio-2.1.10/

    # ./configure

    # make

    # make install


    4. Run fio -h.

    #  fio -h


    fio [options] [job options] <job file(s)>

      --debug=options Enable debug logging. May be one/more of:



      --parse-only Parse options only, don't start any IO

      --output Write output to file

      --runtime Runtime in seconds

      --latency-log Generate per-job latency logs

      --bandwidth-log Generate per-job bandwidth logs

      --minimal Minimal (terse) output

      --output-format=x Output format (terse,json,normal)

      --terse-version=x Set terse version output format to 'x'

      --version Print version info and exit

      --help Print this page

      --cpuclock-test Perform test/validation of CPU clock

      --crctest Test speed of checksum functions

      --cmdhelp=cmd Print command help, "all" for all of them

      --enghelp=engine Print ioengine help, or list available ioengines

      --enghelp=engine,cmd Print help for an ioengine cmd

      --showcmd Turn a job file into command line options

      --eta=when When ETA estimate should be printed

                May be "always", "never" or "auto"

      --eta-newline=time Force a new line for every 'time' period passed

      --status-interval=t Force full status dump every 't' period passed

      --readonly Turn on safety read-only checks, preventing writes

      --section=name Only run specified section in job file

      --alloc-size=kb Set smalloc pool to this size in kb (def 1024)

      --warnings-fatal Fio parser warnings are fatal

      --max-jobs=nr Maximum number of threads/processes to support

      --server=args Start a backend fio server

      --daemonize=pidfile Background fio server, write pid to file

      --client=hostname Talk to remote backend fio server at hostname

      --idle-prof=option Report cpu idleness on a system or percpu basis

      (option=system,percpu) or run unit work

      calibration only (option=calibrate)


    5. Run fio testing benchmark.


    Note: It is recommended to set the following parameters for high throughput.

    • --numjobs=16 <number of cores>: Check the number of cores in the host using lscpu

    # lscpu

    Architecture:          x86_64

    CPU op-mode(s):        32-bit, 64-bit

    Byte Order:            Little Endian

    CPU(s):                16

    On-line CPU(s) list:   0-15

    Thread(s) per core:    2

    Core(s) per socket:    8

    Socket(s):             1

    NUMA node(s):          1

    Vendor ID:             GenuineIntel

    CPU family:            6

    Model:                 62

    Model name:            Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz

    Stepping:              4

    CPU MHz:               1200.561

    BogoMIPS:              4004.05

    Virtualization:        VT-x

    L1d cache:             32K

    L1i cache:             32K

    L2 cache:              256K

    L3 cache:              20480K

    NUMA node0 CPU(s):     0-15


    • --bs=64k <block size>: Block size. Use higher block size for better throughput.


    • --filename=/dev/nvme0n1<device location>: This is the block device used. In this case, it is an NVMe device. In other cases, it could be any other local or remote device.


    For other parameters, refer to the fio man page.


    Here is an example for using ConnectX-4 Lx adapters:

    fio --bs=64k --numjobs=16 --iodepth=4 --loops=1 --direct=1 --invalidate=1 --fsync_on_close=1 --randrepeat=1 --norandommap --time_based --runtime=60 --filename=/dev/nvme0n1  --name=read-phase --rw=randread

    read-phase: (g=0): rw=randread, bs=64K-64K/64K-64K/64K-64K, ioengine=sync, iodepth=4



    Starting 16 processes

    Jobs: 16 (f=16): [rrrrrrrrrrrrrrrr] [18.0% done] [2722MB/0KB/0KB /s] [43.6K/0/0 iops] [eta 00m:50s]