[netperf-talk] How can I get a designated throughput by netperf

[Rick Jones]

I think this is a good topic for the netperf-talk mailing list, so I
will take the liberty of cc'ing it. Alas my mail client (Thunderbird)
seems to have eaten spaces in the original, messing-up the formatting
considerably.  (Suggestions about Thudnerbird settings to address this
would be welcome.)

On 07/20/2011 12:06 AM, Haifeng Li wrote:
> Hi,
> I am investigating netperf and very interested with it.
> I am conducting performance test in my Lab, which is maintain a 
> designated throughput by netperf.I use following commands and hope to 
> burst 800mbps network throughput by netperf, as we wish, it works 
> fine(Case 1).
> but sometimes we only got *120.98mbps* throughput and we use the command 
> same as *799.98mbps*(Case 2).
> Can anyone tell me why was it? or is there any better command/method/way 
> to do this?
> Thanks and regards,
> Haifeng
> *Case 1:*
> # *netperf -t TCP_STREAM -H 192.168.2.252 -b 100 -w 1000 -D 2 -l 60 -f m 
> -c -C -- -m 1000k
> *TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.252 
> (192.168.2.252) port 0 AF_INET : histogram : interval : demo
> Interim result: 801.19 10^6bits/s over 2.06 seconds
> Interim result: 805.87 10^6bits/s over 2.01 seconds
> Interim result: 792.06 10^6bits/s over 2.03 seconds
> Interim result: 798.99 10^6bits/s over 2.01 seconds
> Interim result: 802.91 10^6bits/s over 2.00 seconds
> Recv Send Send Utilization Service Demand
> Socket Socket Message Elapsed Send Recv Send Recv
> Size Size Size Time Throughput local remote local remote
> bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
> 87380 16384 1000000 60.00 *799.98* 6.77 12.81 1.387 2.625
> *Case 2:*
> # *netperf -t TCP_STREAM -H 192.168.2.252 -b 100 -w 1000 -D 2 -l 60 -f m 
> -c -C -- -m 1000k
> *TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.252 
> (192.168.2.252) port 0 AF_INET : histogram : interval : demo
> Interim result: 121.19 10^6bits/s over 2.06 seconds
> Interim result: 119.87 10^6bits/s over 2.01 seconds
> Interim result: 123.06 10^6bits/s over 2.03 seconds
> Interim result: 120.99 10^6bits/s over 2.01 seconds
> Interim result: 123.91 10^6bits/s over 2.00 seconds
> Recv Send Send Utilization Service Demand
> Socket Socket Message Elapsed Send Recv Send Recv
> Size Size Size Time Throughput local remote local remote
> bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
> 87380 16384 1000000 60.00 *120.98* 6.77 12.81 1.387 2.625

Do you really want a burst of 800ish mbits once a second, or are you
really interested in simply 800ish mbit/s overall?  Sending 800 million
bits at once (-m 1000k with a burst of 100) is quite bursty indeed.   I
myself would probably go with much smaller, more frequent bursts.
Perhaps even:

netperf -t TCP_STREAM -H 192.168.2.2 -b 1 -w 10 -D 2 -l 60 -f m -c -C --
-m 1000k

Things that might be happening - TCP retransmissions keeping the
congestion window small, the congestion window resetting after idle
(iirc that happens after the connection has been idle for one RTO).  The
autotuning may not be growing the window sufficiently for you depending
on the RTT of your network.  You could try a fixed socket buffer size
large enough to hold each burst.

One other really interesting thing appears in the netperf output though,
which I cannot explain.  In both cases, the CPU utilization and service
demands are exactly the same, which I find quite surprising indeed.  I
could see the service demands being similar at different throughputs,
but I cannot see where the CPU utilization would be the same at both
sides at different throughputs.  Particularly the receiving side.  I
could see the sending side CPU utilization being the same if you had
added --enable-spin to the ./configure command, but then the throughput
drop should have caused a massive increase in service demand on the
sending side.

happy benchmarking,

rick jones