How to test the network speed/throughput between two Linux servers

ADVERTISEMENTS



iperf is a tool for performing network throughput measurements. It can test either TCP or UDP throughput. To perform an iperf test the user must establish both a server (to discard traffic) and a client (to generate traffic).

I have setup total four servers in cloud based data center. They run either CentOS or Ubuntu Linux. I need to make sure clustered file system performance is reasonable as my servers shared with other users. How do I check the speed between the two Linux servers using command line options for private LAN/VLAN?You can test the network speed (throughput) between Ubuntu/CentOS/Debian/Fedora or Unix box using the iperf command. Iperf commands show info about bandwidth, delay, jitter, and datagram loss. From the man page:

Here is my sample setup:



Step 1 – Installation

You need to install iperf as per your Linux distro version. There are two versions of iperf:

iperf iperf3

The iperf/iperf3 works on Linux, Unix, Windows, *BSD and other operating systems.

Install iperf on a Ubuntu/Debian Linux server

Type the following apt-get command/[nixmcd name=”apt”] to install iperf on both servers:

$ sudo apt-get update

$ sudo apt-get install iperf

Sample outputs:

Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: iperf 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 51.0 kB of archives. After this operation, 184 kB of additional disk space will be used. Get: 1 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf amd64 2.0.5+dfsg1- 2 [ 51.0 kB ] Fetched 51.0 kB in 0s ( 4921 kB/s ) Selecting previously unselected package iperf. ( Reading database ... 26108 files and directories currently installed. ) Preparing to unpack .../iperf_2.0.5+dfsg1- 2 _amd64.deb ... Unpacking iperf ( 2.0.5+dfsg1- 2 ) ... Processing triggers for man-db ( 2.7.5- 1 ) ... Setting up iperf ( 2.0.5+dfsg1- 2 ) ... Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: iperf 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 51.0 kB of archives. After this operation, 184 kB of additional disk space will be used. Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf amd64 2.0.5+dfsg1-2 [51.0 kB] Fetched 51.0 kB in 0s (4921 kB/s) Selecting previously unselected package iperf. (Reading database ... 26108 files and directories currently installed.) Preparing to unpack .../iperf_2.0.5+dfsg1-2_amd64.deb ... Unpacking iperf (2.0.5+dfsg1-2) ... Processing triggers for man-db (2.7.5-1) ... Setting up iperf (2.0.5+dfsg1-2) ...

A note about installing iperf3 on a Debian/Ubuntu Linux

It is also possible to install iperf3 (latest version) as follows:

$ sudo apt-get install iperf3

Sample outputs:

Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libiperf0 The following NEW packages will be installed: iperf3 libiperf0 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 58.5 kB of archives. After this operation, 238 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://mirrors.linode.com/ubuntu xenial/universe amd64 libiperf0 amd64 3.0.11-1 [50.4 kB] Get:2 http://mirrors.linode.com/ubuntu xenial/universe amd64 iperf3 amd64 3.0.11-1 [8090 B] Fetched 58.5 kB in 0s (4796 kB/s) Selecting previously unselected package libiperf0:amd64. (Reading database ... 26120 files and directories currently installed.) Preparing to unpack .../libiperf0_3.0.11-1_amd64.deb ... Unpacking libiperf0:amd64 (3.0.11-1) ... Selecting previously unselected package iperf3. Preparing to unpack .../iperf3_3.0.11-1_amd64.deb ... Unpacking iperf3 (3.0.11-1) ... Processing triggers for man-db (2.7.5-1) ... Setting up libiperf0:amd64 (3.0.11-1) ... Setting up iperf3 (3.0.11-1) ... Processing triggers for libc-bin (2.23-0ubuntu9) ...

Install iperf on a Fedora Linux server

Type the following [nixmcd name=”dnf”] to install iperf on both servers:

$ sudo dnf install iperf

A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux

Type the following command:

$ sudo dnf install iperf3

Install iperf on a CentOS/RHEL/Oracle/Scientific Linux server

Type the following [nixmcd name=”yum”] to install iperf on both servers (first turn on EPEL repo for CentOS/RHEL):

$ sudo yum install iperf

Sample outputs:

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: centos.mirror.snu.edu.in * epel: mirror01.idc.hinet.net * extras: centos.mirror.snu.edu.in * updates: centos.mirror.snu.edu.in Resolving Dependencies --> Running transaction check ---> Package iperf.x86_64 0 :2.0.8- 1 .el7 will be installed --> Finished Dependency Resolution Dependencies Resolved = ==================================================================== Package Arch Version Repository Size = ==================================================================== Installing: iperf x86_64 2.0.8- 1 .el7 epel 321 k Transaction Summary = ==================================================================== Install 1 Package Total download size: 321 k Installed size: 396 k Is this ok [ y/d/N ] : y Downloading packages: iperf-2.0.8- 1 .el7.x86_64.rpm | 321 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : iperf-2.0.8- 1 .el7.x86_64 1 / 1 Verifying : iperf-2.0.8- 1 .el7.x86_64 1 / 1 Installed: iperf.x86_64 0 :2.0.8- 1 .el7 Complete! Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: centos.mirror.snu.edu.in * epel: mirror01.idc.hinet.net * extras: centos.mirror.snu.edu.in * updates: centos.mirror.snu.edu.in Resolving Dependencies --> Running transaction check ---> Package iperf.x86_64 0:2.0.8-1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ===================================================================== Package Arch Version Repository Size ===================================================================== Installing: iperf x86_64 2.0.8-1.el7 epel 321 k Transaction Summary ===================================================================== Install 1 Package Total download size: 321 k Installed size: 396 k Is this ok [y/d/N]: y Downloading packages: iperf-2.0.8-1.el7.x86_64.rpm | 321 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : iperf-2.0.8-1.el7.x86_64 1/1 Verifying : iperf-2.0.8-1.el7.x86_64 1/1 Installed: iperf.x86_64 0:2.0.8-1.el7 Complete!

A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux

Type the following command:

$ sudo yum install iperf3

Step 2 – Start an iperf server on serverA (server mode)

Type the following command to run iperf in server mode:

iperf -s

OR

iperf3 -s

Sample outputs:

------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------

A note about firewall config

Please note that you must open TCP port 5001 on the serverA. See how to use ufw to open a port on Ubuntu/Debian Linux:

$ sudo ufw allow from 192.168.149.0/24 to 192.168.149.69 port 5001 proto tcp

It is also possible to change the port by passing the -p option as follows (e.g. open and use TCP port 2323):

iperf -s -p 2323

Step 3 – Start an iperf server on serverB (client mode)

Type the following command to run in client mode, connecting to 192.168.149.69:

iperf -c {ip-address-of-serverA-here}

iperf -c {ip-address-of-serverA-here} -p {tcp-port}

iperf -c 192.168.149.69 -p 2323

iperf -c 192.168.149.69

OR

iperf3 -c {ip-address-of-serverA-here}

iperf3 -c {ip-address-of-serverA-here} -p {tcp-port}

iperf3 -c 192.168.149.69 -p 2323

iperf3 -c 192.168.149.69

Sample outputs:



How to use UDP rather than TCP

On the server side:

iperf -s -u

On the client side:

iperf -c 192.168.149.69 -u

See also: How To Install Speedtest-cli On a CentOS / RHEL / Scientific / Fedora Linux To Check Internet Speed From the Bash Shelll

How to set number of parallel client threads to run (client mode options)

Pass the -P option:

iperf -c 192.168.149.69 -P 3

You can also set time in seconds to transmit for (default 10 secs) as follows:

iperf -c 192.168.149.69 -P 3 -t 30

Sample outputs:

------------------------------------------------------------ Client connecting to 192.168.149.69, TCP port 5001 TCP window size: 85.0 KByte ( default ) ------------------------------------------------------------ [ 5 ] local 192.168.181.40 port 54198 connected with 192.168.149.69 port 5001 [ 3 ] local 192.168.181.40 port 54194 connected with 192.168.149.69 port 5001 [ 4 ] local 192.168.181.40 port 54196 connected with 192.168.149.69 port 5001 [ ID ] Interval Transfer Bandwidth [ 5 ] 0.0 - 30.0 sec 2.26 GBytes 647 Mbits/sec [ 3 ] 0.0 - 30.0 sec 2.89 GBytes 828 Mbits/sec [ 4 ] 0.0 - 30.5 sec 1.91 GBytes 539 Mbits/sec [ SUM ] 0.0 - 30.5 sec 7.07 GBytes 1.99 Gbits/sec ------------------------------------------------------------ Client connecting to 192.168.149.69, TCP port 5001 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 5] local 192.168.181.40 port 54198 connected with 192.168.149.69 port 5001 [ 3] local 192.168.181.40 port 54194 connected with 192.168.149.69 port 5001 [ 4] local 192.168.181.40 port 54196 connected with 192.168.149.69 port 5001 [ ID] Interval Transfer Bandwidth [ 5] 0.0-30.0 sec 2.26 GBytes 647 Mbits/sec [ 3] 0.0-30.0 sec 2.89 GBytes 828 Mbits/sec [ 4] 0.0-30.5 sec 1.91 GBytes 539 Mbits/sec [SUM] 0.0-30.5 sec 7.07 GBytes 1.99 Gbits/sec

Here is output from iperf3:

## on the server side ##

iperf3 -s

## on the client side ##

iperf3 -c 192.168.149.69



Sample outputs:

Connecting to host 192.168.149.69, port 5201 [ 4 ] local 192.168.181.40 port 55522 connected to 192.168.149.69 port 5201 [ ID ] Interval Transfer Bandwidth Retr Cwnd [ 4 ] 0.00 - 1.00 sec 235 MBytes 1.97 Gbits/sec 32 609 KBytes [ 4 ] 1.00 - 2.00 sec 230 MBytes 1.93 Gbits/sec 21 670 KBytes [ 4 ] 2.00 - 3.00 sec 159 MBytes 1.33 Gbits/sec 19 635 KBytes [ 4 ] 3.00 - 4.00 sec 235 MBytes 1.97 Gbits/sec 7 686 KBytes [ 4 ] 4.00 - 5.00 sec 229 MBytes 1.92 Gbits/sec 86 757 KBytes [ 4 ] 5.00 - 6.00 sec 206 MBytes 1.73 Gbits/sec 6 537 KBytes [ 4 ] 6.00 - 7.00 sec 210 MBytes 1.76 Gbits/sec 5 556 KBytes [ 4 ] 7.00 - 8.00 sec 235 MBytes 1.97 Gbits/sec 10 617 KBytes [ 4 ] 8.00 - 9.00 sec 245 MBytes 2.06 Gbits/sec 11 670 KBytes [ 4 ] 9.00 - 10.00 sec 234 MBytes 1.96 Gbits/sec 10 713 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID ] Interval Transfer Bandwidth Retr [ 4 ] 0.00 - 10.00 sec 2.17 GBytes 1.86 Gbits/sec 207 sender [ 4 ] 0.00 - 10.00 sec 2.16 GBytes 1.86 Gbits/sec receiver Connecting to host 192.168.149.69, port 5201 [ 4] local 192.168.181.40 port 55522 connected to 192.168.149.69 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 235 MBytes 1.97 Gbits/sec 32 609 KBytes [ 4] 1.00-2.00 sec 230 MBytes 1.93 Gbits/sec 21 670 KBytes [ 4] 2.00-3.00 sec 159 MBytes 1.33 Gbits/sec 19 635 KBytes [ 4] 3.00-4.00 sec 235 MBytes 1.97 Gbits/sec 7 686 KBytes [ 4] 4.00-5.00 sec 229 MBytes 1.92 Gbits/sec 86 757 KBytes [ 4] 5.00-6.00 sec 206 MBytes 1.73 Gbits/sec 6 537 KBytes [ 4] 6.00-7.00 sec 210 MBytes 1.76 Gbits/sec 5 556 KBytes [ 4] 7.00-8.00 sec 235 MBytes 1.97 Gbits/sec 10 617 KBytes [ 4] 8.00-9.00 sec 245 MBytes 2.06 Gbits/sec 11 670 KBytes [ 4] 9.00-10.00 sec 234 MBytes 1.96 Gbits/sec 10 713 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 2.17 GBytes 1.86 Gbits/sec 207 sender [ 4] 0.00-10.00 sec 2.16 GBytes 1.86 Gbits/sec receiver

Getting help

Type the following command:

$ man iperf

$ iperf --help

Usage: iperf [ -s|-c host ] [ options ] iperf [ -h|--help ] [ -v|--version ] Client/Server: -f, --format [ kmKM ] format to report: Kbits, Mbits, KBytes, MBytes -i, --interval # seconds between periodic bandwidth reports -l, --len # [ KM ] length of buffer to read or write ( default 8 KB ) -m, --print_mss print TCP maximum segment size ( MTU - TCP/IP header ) -o, --output <filename> output the report or error message to this specified file -p, --port # server port to listen on/connect to -u, --udp use UDP rather than TCP -w, --window # [ KM ] TCP window size ( socket buffer size ) -B, --bind <host> bind to <host>, an interface or multicast address -C, --compatibility for use with older versions does not sent extra msgs -M, --mss # set TCP maximum segment size ( MTU - 40 bytes ) -N, --nodelay set TCP no delay, disabling Nagle's Algorithm -V, --IPv6Version Set the domain to IPv6 Server specific: -s, --server run in server mode -U, --single_udp run in single threaded UDP mode -D, --daemon run the server as a daemon Client specific: -b, --bandwidth # [ KM ] for UDP, bandwidth to send at in bits/sec ( default 1 Mbit/sec, implies -u ) -c, --client <host> run in client mode, connecting to <host> -d, --dualtest Do a bidirectional test simultaneously -n, --num # [ KM ] number of bytes to transmit ( instead of -t ) -r, --tradeoff Do a bidirectional test individually -t, --time # time in seconds to transmit for ( default 10 secs ) -F, --fileinput <name> input the data to be transmitted from a file -I, --stdin input the data to be transmitted from stdin -L, --listenport # port to receive bidirectional tests back on -P, --parallel # number of parallel client threads to run -T, --ttl # time-to-live, for multicast ( default 1 ) -Z, --linux-congestion <algo> set TCP congestion control algorithm ( Linux only ) Miscellaneous: -x, --reportexclude [ CDMSV ] exclude C ( connection ) D ( data ) M ( multicast ) S ( settings ) V ( server ) reports -y, --reportstyle C report as a Comma-Separated Values -h, --help print this message and quit -v, --version print version information and quit [ KM ] Indicates options that support a K or M suffix for kilo- or mega- The TCP window size option can be set by the environment variable TCP_WINDOW_SIZE. Most other options can be set by an environment variable IPERF_<long option name>, such as IPERF_BANDWIDTH. Usage: iperf [-s|-c host] [options] iperf [-h|--help] [-v|--version] Client/Server: -f, --format [kmKM] format to report: Kbits, Mbits, KBytes, MBytes -i, --interval # seconds between periodic bandwidth reports -l, --len #[KM] length of buffer to read or write (default 8 KB) -m, --print_mss print TCP maximum segment size (MTU - TCP/IP header) -o, --output <filename> output the report or error message to this specified file -p, --port # server port to listen on/connect to -u, --udp use UDP rather than TCP -w, --window #[KM] TCP window size (socket buffer size) -B, --bind <host> bind to <host>, an interface or multicast address -C, --compatibility for use with older versions does not sent extra msgs -M, --mss # set TCP maximum segment size (MTU - 40 bytes) -N, --nodelay set TCP no delay, disabling Nagle's Algorithm -V, --IPv6Version Set the domain to IPv6 Server specific: -s, --server run in server mode -U, --single_udp run in single threaded UDP mode -D, --daemon run the server as a daemon Client specific: -b, --bandwidth #[KM] for UDP, bandwidth to send at in bits/sec (default 1 Mbit/sec, implies -u) -c, --client <host> run in client mode, connecting to <host> -d, --dualtest Do a bidirectional test simultaneously -n, --num #[KM] number of bytes to transmit (instead of -t) -r, --tradeoff Do a bidirectional test individually -t, --time # time in seconds to transmit for (default 10 secs) -F, --fileinput <name> input the data to be transmitted from a file -I, --stdin input the data to be transmitted from stdin -L, --listenport # port to receive bidirectional tests back on -P, --parallel # number of parallel client threads to run -T, --ttl # time-to-live, for multicast (default 1) -Z, --linux-congestion <algo> set TCP congestion control algorithm (Linux only) Miscellaneous: -x, --reportexclude [CDMSV] exclude C(connection) D(data) M(multicast) S(settings) V(server) reports -y, --reportstyle C report as a Comma-Separated Values -h, --help print this message and quit -v, --version print version information and quit [KM] Indicates options that support a K or M suffix for kilo- or mega- The TCP window size option can be set by the environment variable TCP_WINDOW_SIZE. Most other options can be set by an environment variable IPERF_<long option name>, such as IPERF_BANDWIDTH.

See iperf home page for more info.