[netperf-talk] How can I get a designated throughput by netperf

[Rick Jones]

On 07/20/2011 06:23 PM, Haifeng Li wrote:
> Hi Rick,
> 
> Thanks so much for your prompt response.
> 
> So did you meet this scenario during the test beforetime? I hope to know is this bug or ...?

Clearly something odd is happening in your test setup, but what exactly
I do not know.  What was the exact ./configure command you used?
Ostensibly, netperf would have included "spin interval" in the test
header had you used --enable-spin, but I'm paranoid :)

Also, can you describe a bit more about your test setup - what sort of
systems, is there any sort of background CPU load (netperf assumes it is
the only thing running when it is measuring CPU utilization, so counts
all CPU used as its own.)

Checking if there are retransmissions during the slow part would be a
good idea.

What sort of NICs are used on either end?  What sort of interrupt
coalescing settings do they have?  The whole bit

rick

> 
> 
> Maybe I should send a mail to thunderbird maillist, ^-^

:)  Nah, although I suspect it was my thunderbird client eating spaces
when it was converting your HTML (?) email to plain text
> 
> 
> 
> Thanks and regards,
> Haifeng
> 
> 
> ----- Original Message -----
> From: "Rick Jones"<rick.jones2 at hp.com>
> To: "Haifeng Li"<lhffjzh at 126.com>
> Cc:<netperf-talk at netperf.org>
> Sent: Thursday, July 21, 2011 2:06 AM
> Subject: Re: How can I get a designated throughput by netperf
> 
> 
>> I think this is a good topic for the netperf-talk mailing list, so I
>> will take the liberty of cc'ing it. Alas my mail client (Thunderbird)
>> seems to have eaten spaces in the original, messing-up the formatting
>> considerably.  (Suggestions about Thudnerbird settings to address this
>> would be welcome.)
>>
>> On 07/20/2011 12:06 AM, Haifeng Li wrote:
>>> Hi,
>>> I am investigating netperf and very interested with it.
>>> I am conducting performance test in my Lab, which is maintain a
>>> designated throughput by netperf.I use following commands and hope to
>>> burst 800mbps network throughput by netperf, as we wish, it works
>>> fine(Case 1).
>>> but sometimes we only got *120.98mbps* throughput and we use the command
>>> same as *799.98mbps*(Case 2).
>>> Can anyone tell me why was it? or is there any better command/method/way
>>> to do this?
>>> Thanks and regards,
>>> Haifeng
>>> *Case 1:*
>>> # *netperf -t TCP_STREAM -H 192.168.2.252 -b 100 -w 1000 -D 2 -l 60 -f m
>>> -c -C -- -m 1000k
>>> *TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.252
>>> (192.168.2.252) port 0 AF_INET : histogram : interval : demo
>>> Interim result: 801.19 10^6bits/s over 2.06 seconds
>>> Interim result: 805.87 10^6bits/s over 2.01 seconds
>>> Interim result: 792.06 10^6bits/s over 2.03 seconds
>>> Interim result: 798.99 10^6bits/s over 2.01 seconds
>>> Interim result: 802.91 10^6bits/s over 2.00 seconds
>>> Recv Send Send Utilization Service Demand
>>> Socket Socket Message Elapsed Send Recv Send Recv
>>> Size Size Size Time Throughput local remote local remote
>>> bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
>>> 87380 16384 1000000 60.00 *799.98* 6.77 12.81 1.387 2.625
>>> *Case 2:*
>>> # *netperf -t TCP_STREAM -H 192.168.2.252 -b 100 -w 1000 -D 2 -l 60 -f m
>>> -c -C -- -m 1000k
>>> *TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.252
>>> (192.168.2.252) port 0 AF_INET : histogram : interval : demo
>>> Interim result: 121.19 10^6bits/s over 2.06 seconds
>>> Interim result: 119.87 10^6bits/s over 2.01 seconds
>>> Interim result: 123.06 10^6bits/s over 2.03 seconds
>>> Interim result: 120.99 10^6bits/s over 2.01 seconds
>>> Interim result: 123.91 10^6bits/s over 2.00 seconds
>>> Recv Send Send Utilization Service Demand
>>> Socket Socket Message Elapsed Send Recv Send Recv
>>> Size Size Size Time Throughput local remote local remote
>>> bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
>>> 87380 16384 1000000 60.00 *120.98* 6.77 12.81 1.387 2.625
>>
>> Do you really want a burst of 800ish mbits once a second, or are you
>> really interested in simply 800ish mbit/s overall?  Sending 800 million
>> bits at once (-m 1000k with a burst of 100) is quite bursty indeed.   I
>> myself would probably go with much smaller, more frequent bursts.
>> Perhaps even:
>>
>> netperf -t TCP_STREAM -H 192.168.2.2 -b 1 -w 10 -D 2 -l 60 -f m -c -C --
>> -m 1000k
>>
>> Things that might be happening - TCP retransmissions keeping the
>> congestion window small, the congestion window resetting after idle
>> (iirc that happens after the connection has been idle for one RTO).  The
>> autotuning may not be growing the window sufficiently for you depending
>> on the RTT of your network.  You could try a fixed socket buffer size
>> large enough to hold each burst.
>>
>> One other really interesting thing appears in the netperf output though,
>> which I cannot explain.  In both cases, the CPU utilization and service
>> demands are exactly the same, which I find quite surprising indeed.  I
>> could see the service demands being similar at different throughputs,
>> but I cannot see where the CPU utilization would be the same at both
>> sides at different throughputs.  Particularly the receiving side.  I
>> could see the sending side CPU utilization being the same if you had
>> added --enable-spin to the ./configure command, but then the throughput
>> drop should have caused a massive increase in service demand on the
>> sending side.
>>
>> happy benchmarking,
>>
>> rick jones