[netperf-talk] Same request, different result between windows and linux

[Rick Jones]

刘亚东 wrote:
> hi, all,
>  
>        I got a very mysterious problem when testing under different OS.
>        Here both size of the receive and send buffer have been set 
> the same between windows and linux, but the results were different. 
> Under windows, the Recv socket and the send socket size are the same as 
> the  size of the receive and send buffer, but under linux, the 
> Recv socket and the send socket size became double. why? Just watch the 
> follows message.
>  
>       I don't konw whether the difference is natural between windows and 
> linux or there is some relationship with the netperf tool?
>       Would somebody like to help me find out the answer?
>  
> ** 
> *Under windows:*
>  >netperf.exe -H HOST_IP -t TCP_STREAM -l 10 -- -m 1024 -s 2048 -S 2048
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to HOST_IP 
> (HOST_IP) port 0 AF_INET
> Recv   Send    Send
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>   2048   2048   1024    10.55     390.65
>  
> *Under linux:*
> # netperf -H HOST_IP -t TCP_STREAM -l 10 -- -m 1024 -s 2048 -S 2048
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to HOST_IP 
> (HOST_IP) port 0 AF_INET
> Recv   Send    Send
> Socket Socket  Message  Elapsed
> Size   Size    Size     Time     Throughput
> bytes  bytes   bytes    secs.    10^6bits/sec
>   4096   4096   1024    10.00     522.79


This is "normal" if annoying behaviour under Linux.  Up to one of the sysctl 
limits (the three entry tcp_[wr]mem_max IIRC) Linux will take the requested 
value in the setsockopt() call, and double it to account for overheads.

I'm guessing that HOST_IP is actually local because I don't think there is much 
chance of getting those performance levels across a GbE link with those tiny 
window sizes - of course, I am ass-u-me-ing that the -S/-s options at those low 
levels are indeed causing window sizes that small - a packet trace capturing the 
SYN segments would need to be taken to confirm it.

happy benchmarking,

rick jones

BTW, it is even more "fun" when you don't use -s/-S and take the default under 
Linux - what classic netperf will report will be the values just before  the 
connection is established, but the autotuning of the socket buffers done by 
linux over the life of the connection means that the connection will actually 
use values rather different from the initial ones.  This is dealt with in the 
"omni" tests by taking a snapshot of the socket buffer sizes at the end of the 
test, just before the data connection is closed.