How to find disk I/O latency with ioping monitoring tool on Linux

ioping

ADVERTISEMENTS



Installation

How do I monitor disk I/O latency in real time on Linux, FreeBSD, OpenBSD, and MacOS operating system?You can usein real time. It shows disk latency in the same way as ping command shows network latency on Linux or Unix-like system.

Install as per your operating system.

Install ioping on a Debian/Ubuntu/Mint Linux

Type the following apt-get command/apt command:

$ sudo apt-get install ioping

Install ioping on an Arch Linux

Type the following pacman command:

$ sudo pacman -S ioping

Install ioping on a Fedora Linux

Type the following dnf command:

$ sudo dnf install ioping

Install ioping on a macOS

Type the following brew command:

$ brew install ioping

Sample outputs:

== > Downloading https://homebrew.bintray.com/bottles/ioping- 1.0 .sierra.bottle.tar.gz ######################################################################## 100.0 % == > Pouring ioping- 1.0 .sierra.bottle.tar.gz * /usr/local/Cellar/ioping/ 1.0 : 7 files, 79.4KB ==> Downloading https://homebrew.bintray.com/bottles/ioping-1.0.sierra.bottle.tar.gz ######################################################################## 100.0% ==> Pouring ioping-1.0.sierra.bottle.tar.gz * /usr/local/Cellar/ioping/1.0: 7 files, 79.4KB

Install ioping on a FreeBSD

Type the following pkg command:

# pkg install ioping

Sample outputs:

Updating FreeBSD repository catalogue... Fetching meta.txz: 100 % 944 B 0.9kB/s 00:01 Fetching packagesite.txz: 100 % 6 MiB 6.0MB/s 00:01 Processing entries: 100 % FreeBSD repository update completed. 26484 packages processed. All repositories are up to date. The following 1 package ( s ) will be affected ( of 0 checked ) : New packages to be INSTALLED: ioping: 1.0 Number of packages to be installed: 1 23 KiB to be downloaded. Proceed with this action? [ y/N ] : y [ 1/1 ] Fetching ioping- 1.0 .txz: 100 % 23 KiB 23.6kB/s 00:01 Checking integrity... done ( 0 conflicting ) [ 1/1 ] Installing ioping- 1.0 ... Extracting ioping- 1.0 : 100 % Updating FreeBSD repository catalogue... Fetching meta.txz: 100% 944 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 6 MiB 6.0MB/s 00:01 Processing entries: 100% FreeBSD repository update completed. 26484 packages processed. All repositories are up to date. The following 1 package(s) will be affected (of 0 checked): New packages to be INSTALLED: ioping: 1.0 Number of packages to be installed: 1 23 KiB to be downloaded. Proceed with this action? [y/N]: y [1/1] Fetching ioping-1.0.txz: 100% 23 KiB 23.6kB/s 00:01 Checking integrity... done (0 conflicting) [1/1] Installing ioping-1.0... Extracting ioping-1.0: 100%

Install ioping on a OpenBSD

Type the following pkg_add command:

# pkg_add ioping

Sample outputs:

quirks-2.304 signed on 2017-04-02T15:01:33Z ioping-1.0: ok

How do I use ioping?

Warning: Use the ioping command with care. Wrong options can destroy your data. Read man page for more info.

The syntax is:

ioping /tmp/

ioping /dev/sda1

ioping [options] device/dir/file

See disk I/O latency using the default values and the current directory

$ ioping .

To stop after 12 count, pass the -c option:

$ ioping -c 12 .

Sample outputs from OpenBSD running on Linode server:

ioping: non-cached I/O not supported by this platform ioping: you can use write I/O to get reliable results 4 KiB <<< . ( ffs /dev/sd0k ) : request = 1 time=15.9 us ( warmup ) 4 KiB <<< . ( ffs /dev/sd0k ) : request = 2 time=22.3 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 3 time=20 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 4 time=19.9 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 5 time=21.9 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 6 time=25.7 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 7 time=24.8 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 8 time=25.3 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 9 time=85.5 us ( slow ) 4 KiB <<< . ( ffs /dev/sd0k ) : request = 10 time=23.9 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 11 time=38.2 us 4 KiB <<< . ( ffs /dev/sd0k ) : request = 12 time=30.8 us --- . ( ffs /dev/sd0k ) ioping statistics --- 11 requests completed in 338.3 us, 44 KiB read, 32.5 k iops, 127.0 MiB/s generated 12 requests in 11.0 s, 48 KiB, 1 iops, 4.36 KiB/s min/avg/max/mdev = 19.9 us / 30.8 us / 85.5 us / 18.0 u ioping: non-cached I/O not supported by this platform ioping: you can use write I/O to get reliable results 4 KiB <<< . (ffs /dev/sd0k): request=1 time=15.9 us (warmup) 4 KiB <<< . (ffs /dev/sd0k): request=2 time=22.3 us 4 KiB <<< . (ffs /dev/sd0k): request=3 time=20 us 4 KiB <<< . (ffs /dev/sd0k): request=4 time=19.9 us 4 KiB <<< . (ffs /dev/sd0k): request=5 time=21.9 us 4 KiB <<< . (ffs /dev/sd0k): request=6 time=25.7 us 4 KiB <<< . (ffs /dev/sd0k): request=7 time=24.8 us 4 KiB <<< . (ffs /dev/sd0k): request=8 time=25.3 us 4 KiB <<< . (ffs /dev/sd0k): request=9 time=85.5 us (slow) 4 KiB <<< . (ffs /dev/sd0k): request=10 time=23.9 us 4 KiB <<< . (ffs /dev/sd0k): request=11 time=38.2 us 4 KiB <<< . (ffs /dev/sd0k): request=12 time=30.8 us --- . (ffs /dev/sd0k) ioping statistics --- 11 requests completed in 338.3 us, 44 KiB read, 32.5 k iops, 127.0 MiB/s generated 12 requests in 11.0 s, 48 KiB, 1 iops, 4.36 KiB/s min/avg/max/mdev = 19.9 us / 30.8 us / 85.5 us / 18.0 u

To see disk seek rate

$ sudo ioping -R /dev/sda

To see disk sequential speed (MiB/s)

$ sudo ioping -RL /dev/rdisk1

Sample outputs:



How to set interval between requests

The default is 1 second. To change pass the -i {interval} option. Set 3 seconds:

$ ioping -i 3 .

$ ioping -i 3 /dev/sdb

How do I use asynchronous I/O?

Pass the -A option:

# ioping -c 6 -A /dev/vda1

Sample outputs:

4 KiB from /dev/vda1 (block device 20.0 GiB): request=1 time=458 us 4 KiB from /dev/vda1 (block device 20.0 GiB): request=2 time=226 us 4 KiB from /dev/vda1 (block device 20.0 GiB): request=3 time=477 us 4 KiB from /dev/vda1 (block device 20.0 GiB): request=4 time=412 us 4 KiB from /dev/vda1 (block device 20.0 GiB): request=5 time=381 us 4 KiB from /dev/vda1 (block device 20.0 GiB): request=6 time=356 us --- /dev/vda1 (block device 20.0 GiB) ioping statistics --- 6 requests completed in 5.00 s, 2.60 k iops, 10.1 MiB/s min/avg/max/mdev = 226 us / 385 us / 477 us / 82 us

How to use cached I/O (no cache flush/drop)

Pass the -C option:

$ ioping -c 6 -C /dev/sdb1

# ioping -c 6 -C .

Sample outputs from FreeBSD server hosted at Linode:

4 KiB <<< . ( zfs zroot/ROOT/default ) : request = 1 time=30.8 us ( warmup ) 4 KiB <<< . ( zfs zroot/ROOT/default ) : request = 2 time=45.3 us 4 KiB <<< . ( zfs zroot/ROOT/default ) : request = 3 time=40.9 us 4 KiB <<< . ( zfs zroot/ROOT/default ) : request = 4 time=57.1 us 4 KiB <<< . ( zfs zroot/ROOT/default ) : request = 5 time=47.5 us 4 KiB <<< . ( zfs zroot/ROOT/default ) : request = 6 time=52.8 us --- . ( zfs zroot/ROOT/default ) ioping statistics --- 5 requests completed in 243.6 us, 20 KiB read, 20.5 k iops, 80.2 MiB/s generated 6 requests in 5.06 s, 24 KiB, 1 iops, 4.74 KiB/s min/avg/max/mdev = 40.9 us / 48.7 us / 57.1 us / 5.65 us 4 KiB <<< . (zfs zroot/ROOT/default): request=1 time=30.8 us (warmup) 4 KiB <<< . (zfs zroot/ROOT/default): request=2 time=45.3 us 4 KiB <<< . (zfs zroot/ROOT/default): request=3 time=40.9 us 4 KiB <<< . (zfs zroot/ROOT/default): request=4 time=57.1 us 4 KiB <<< . (zfs zroot/ROOT/default): request=5 time=47.5 us 4 KiB <<< . (zfs zroot/ROOT/default): request=6 time=52.8 us --- . (zfs zroot/ROOT/default) ioping statistics --- 5 requests completed in 243.6 us, 20 KiB read, 20.5 k iops, 80.2 MiB/s generated 6 requests in 5.06 s, 24 KiB, 1 iops, 4.74 KiB/s min/avg/max/mdev = 40.9 us / 48.7 us / 57.1 us / 5.65 us

How to use direct I/O (O_DIRECT)

Pass the -D option:

# ioping -c 6 -D /tmp

Sample outputs from FreeBSD+ZFS based fs:

4 KiB <<< /tmp ( zfs zroot/tmp ) : request = 1 time=19.9 us ( warmup ) 4 KiB <<< /tmp ( zfs zroot/tmp ) : request = 2 time=46.9 us 4 KiB <<< /tmp ( zfs zroot/tmp ) : request = 3 time=87.4 us 4 KiB <<< /tmp ( zfs zroot/tmp ) : request = 4 time=49.7 us 4 KiB <<< /tmp ( zfs zroot/tmp ) : request = 5 time=60.1 us 4 KiB <<< /tmp ( zfs zroot/tmp ) : request = 6 time=50.5 us --- /tmp ( zfs zroot/tmp ) ioping statistics --- 5 requests completed in 294.6 us, 20 KiB read, 17.0 k iops, 66.3 MiB/s generated 6 requests in 5.06 s, 24 KiB, 1 iops, 4.74 KiB/s min/avg/max/mdev = 46.9 us / 58.9 us / 87.4 us / 14.9 us 4 KiB <<< /tmp (zfs zroot/tmp): request=1 time=19.9 us (warmup) 4 KiB <<< /tmp (zfs zroot/tmp): request=2 time=46.9 us 4 KiB <<< /tmp (zfs zroot/tmp): request=3 time=87.4 us 4 KiB <<< /tmp (zfs zroot/tmp): request=4 time=49.7 us 4 KiB <<< /tmp (zfs zroot/tmp): request=5 time=60.1 us 4 KiB <<< /tmp (zfs zroot/tmp): request=6 time=50.5 us --- /tmp (zfs zroot/tmp) ioping statistics --- 5 requests completed in 294.6 us, 20 KiB read, 17.0 k iops, 66.3 MiB/s generated 6 requests in 5.06 s, 24 KiB, 1 iops, 4.74 KiB/s min/avg/max/mdev = 46.9 us / 58.9 us / 87.4 us / 14.9 us

How to use write I/O (please read manpage)

Pass the -W option. From the man page:

Safe for directory target. Write I/O gives more reliable results for systems where non-cached reads are not supported or cached at some level. Might be *DANGEROUS* for file/device: it will shred your data. In this case should be repeated tree times (-WWW).

$ ioping -c 6 -W /tmp

Sample outputs from OpenBSD sever at Linode:

4 KiB >>> /tmp ( ffs /dev/sd0d ) : request = 1 time=363.9 us ( warmup ) 4 KiB >>> /tmp ( ffs /dev/sd0d ) : request = 2 time=684.4 us 4 KiB >>> /tmp ( ffs /dev/sd0d ) : request = 3 time=6.07 ms 4 KiB >>> /tmp ( ffs /dev/sd0d ) : request = 4 time=2.85 ms 4 KiB >>> /tmp ( ffs /dev/sd0d ) : request = 5 time=7.98 ms 4 KiB >>> /tmp ( ffs /dev/sd0d ) : request = 6 time=1.33 ms --- /tmp ( ffs /dev/sd0d ) ioping statistics --- 5 requests completed in 18.9 ms, 20 KiB written, 264 iops, 1.03 MiB/s generated 6 requests in 5.01 s, 24 KiB, 1 iops, 4.79 KiB/s min/avg/max/mdev = 684.4 us / 3.78 ms / 7.98 ms / 2.80 ms 4 KiB >>> /tmp (ffs /dev/sd0d): request=1 time=363.9 us (warmup) 4 KiB >>> /tmp (ffs /dev/sd0d): request=2 time=684.4 us 4 KiB >>> /tmp (ffs /dev/sd0d): request=3 time=6.07 ms 4 KiB >>> /tmp (ffs /dev/sd0d): request=4 time=2.85 ms 4 KiB >>> /tmp (ffs /dev/sd0d): request=5 time=7.98 ms 4 KiB >>> /tmp (ffs /dev/sd0d): request=6 time=1.33 ms --- /tmp (ffs /dev/sd0d) ioping statistics --- 5 requests completed in 18.9 ms, 20 KiB written, 264 iops, 1.03 MiB/s generated 6 requests in 5.01 s, 24 KiB, 1 iops, 4.79 KiB/s min/avg/max/mdev = 684.4 us / 3.78 ms / 7.98 ms / 2.80 ms

How to use read-write ping-pong mode

Pass the -G option:

# ioping -c 10 -G /tmp

Sample outputs Google cloud VM:

4 KiB >>> /tmp ( ext4 /dev/sda1 ) : request = 1 time=614.4 us ( warmup ) 4 KiB <<< /tmp ( ext4 /dev/sda1 ) : request = 2 time=2.11 ms 4 KiB >>> /tmp ( ext4 /dev/sda1 ) : request = 3 time=1.05 ms 4 KiB <<< /tmp ( ext4 /dev/sda1 ) : request = 4 time=1.89 ms 4 KiB >>> /tmp ( ext4 /dev/sda1 ) : request = 5 time=965.0 us 4 KiB <<< /tmp ( ext4 /dev/sda1 ) : request = 6 time=908.3 us 4 KiB >>> /tmp ( ext4 /dev/sda1 ) : request = 7 time=1.17 ms 4 KiB <<< /tmp ( ext4 /dev/sda1 ) : request = 8 time=816.1 us ( fast ) 4 KiB >>> /tmp ( ext4 /dev/sda1 ) : request = 9 time=1.14 ms 4 KiB <<< /tmp ( ext4 /dev/sda1 ) : request = 10 time=2.75 ms ( slow ) --- /tmp ( ext4 /dev/sda1 ) ioping statistics --- 9 requests completed in 12.8 ms, 36 KiB, 703 iops, 2.75 MiB/s generated 10 requests in 9.00 s, 40 KiB, 1 iops, 4.44 KiB/s min/avg/max/mdev = 816.1 us / 1.42 ms / 2.75 ms / 630.4 us 4 KiB >>> /tmp (ext4 /dev/sda1): request=1 time=614.4 us (warmup) 4 KiB <<< /tmp (ext4 /dev/sda1): request=2 time=2.11 ms 4 KiB >>> /tmp (ext4 /dev/sda1): request=3 time=1.05 ms 4 KiB <<< /tmp (ext4 /dev/sda1): request=4 time=1.89 ms 4 KiB >>> /tmp (ext4 /dev/sda1): request=5 time=965.0 us 4 KiB <<< /tmp (ext4 /dev/sda1): request=6 time=908.3 us 4 KiB >>> /tmp (ext4 /dev/sda1): request=7 time=1.17 ms 4 KiB <<< /tmp (ext4 /dev/sda1): request=8 time=816.1 us (fast) 4 KiB >>> /tmp (ext4 /dev/sda1): request=9 time=1.14 ms 4 KiB <<< /tmp (ext4 /dev/sda1): request=10 time=2.75 ms (slow) --- /tmp (ext4 /dev/sda1) ioping statistics --- 9 requests completed in 12.8 ms, 36 KiB, 703 iops, 2.75 MiB/s generated 10 requests in 9.00 s, 40 KiB, 1 iops, 4.44 KiB/s min/avg/max/mdev = 816.1 us / 1.42 ms / 2.75 ms / 630.4 us

How to use use sync I/O (O_SYNC)

Pass the -Y option:

# ioping -c 10 -Y /dev/sdb

Sample outputs from Google compute VM:

4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 1 time=237.5 us ( warmup ) 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 2 time=245.9 us 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 3 time=237.1 us 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 4 time=256.0 us 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 5 time=251.6 us 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 6 time=14.5 ms 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 7 time=252.6 us ( fast ) 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 8 time=318.4 us ( fast ) 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 9 time=260.5 us ( fast ) 4 KiB <<< /dev/sdb ( block device 200 GiB ) : request = 10 time=277.2 us ( fast ) --- /dev/sdb ( block device 200 GiB ) ioping statistics --- 9 requests completed in 16.6 ms, 36 KiB read, 541 iops, 2.12 MiB/s generated 10 requests in 9.00 s, 40 KiB, 1 iops, 4.44 KiB/s min/avg/max/mdev = 237.1 us / 1.85 ms / 14.5 ms / 4.48 ms 4 KiB <<< /dev/sdb (block device 200 GiB): request=1 time=237.5 us (warmup) 4 KiB <<< /dev/sdb (block device 200 GiB): request=2 time=245.9 us 4 KiB <<< /dev/sdb (block device 200 GiB): request=3 time=237.1 us 4 KiB <<< /dev/sdb (block device 200 GiB): request=4 time=256.0 us 4 KiB <<< /dev/sdb (block device 200 GiB): request=5 time=251.6 us 4 KiB <<< /dev/sdb (block device 200 GiB): request=6 time=14.5 ms 4 KiB <<< /dev/sdb (block device 200 GiB): request=7 time=252.6 us (fast) 4 KiB <<< /dev/sdb (block device 200 GiB): request=8 time=318.4 us (fast) 4 KiB <<< /dev/sdb (block device 200 GiB): request=9 time=260.5 us (fast) 4 KiB <<< /dev/sdb (block device 200 GiB): request=10 time=277.2 us (fast) --- /dev/sdb (block device 200 GiB) ioping statistics --- 9 requests completed in 16.6 ms, 36 KiB read, 541 iops, 2.12 MiB/s generated 10 requests in 9.00 s, 40 KiB, 1 iops, 4.44 KiB/s min/avg/max/mdev = 237.1 us / 1.85 ms / 14.5 ms / 4.48 ms

How to use use data sync I/O (O_DSYNC)

Pass the -y option:

$ ioping -c 10 -y .

Sample outputs from macos:



Getting help

Read man page:

$ man ioping

OR

$ ioping -h

Sample outputs:

-c <count> stop after <count> requests -i <interval> interval between requests ( 1s ) -l <speed> speed limit in bytes per second -t <time> minimal valid request time ( 0us ) -T <time> maximum valid request time -s <size> request size ( 4k ) -S <wsize> working set size ( 1m ) -o <offset> working set offset ( 0 ) -w <deadline> stop after <deadline> time passed -p <period> print raw statistics for every <period> requests -P <period> print raw statistics for every <period> in time -A use asynchronous I/O -C use cached I/O ( no cache flush/drop ) -B print final statistics in raw format -D use direct I/O ( O_DIRECT ) -R seek rate test -L use sequential operations -W use write I/O ( please read manpage ) -G read-write ping-pong mode -Y use sync I/O ( O_SYNC ) -y use data sync I/O ( O_DSYNC ) -k keep and reuse temporary file ( ioping.tmp ) -q suppress human-readable output -h display this message and exit -v display version and exit -c <count> stop after <count> requests -i <interval> interval between requests (1s) -l <speed> speed limit in bytes per second -t <time> minimal valid request time (0us) -T <time> maximum valid request time -s <size> request size (4k) -S <wsize> working set size (1m) -o <offset> working set offset (0) -w <deadline> stop after <deadline> time passed -p <period> print raw statistics for every <period> requests -P <period> print raw statistics for every <period> in time -A use asynchronous I/O -C use cached I/O (no cache flush/drop) -B print final statistics in raw format -D use direct I/O (O_DIRECT) -R seek rate test -L use sequential operations -W use write I/O (please read manpage) -G read-write ping-pong mode -Y use sync I/O (O_SYNC) -y use data sync I/O (O_DSYNC) -k keep and reuse temporary file (ioping.tmp) -q suppress human-readable output -h display this message and exit -v display version and exit

See ioping home page for more info.