[netperf-talk] netperf2.4.4 on linux, socket size issue ?

[mark wagner]

Hi Drew

Thanks for the quick response. I'm actually working on a paper on 10G 
tuning so things are at the defaults now so I can show the effects of 
the changes.  More comments in-line.

Andrew Gallatin wrote:
> mark wagner writes:
>  > above.  So, is the drop in performance because I'm really only using an 
>  > 8K buffer even though its getting reported as 16K or is something else 
>  > going on here that I'm oblivious to?
>
> The doubling is a red herring.  By default, linux will auto-tune the
> socket buffer sizes.  If the application sets the socket buffer size,
> this auto-tuning is disabled.   So in your -- -s 8K example, you're
> disabling the auto-tuning, and throttling transmit.
>
>   
So even though netperf reports the message and buffer sizes to be the 
same in both runs, should I assume that those are just incorrect and 
have no bearing on what is really going on?

Please keep in mind that I need to be able to present these results and 
compare / contrast different values. So maybe its best for me to 
explicitly specify the sizes on every run and report what I said to use 
and the throughput rather than use what netperf said the buffers were ?

> BTW, there are some other settings you should tweak for good 10GbE
> performance (like increasing the default socket buffer limits).  
>
> Try adding the following lines to
> /etc/sysctl.conf and execute the command "sysctl -p /etc/sysctl.conf".
>
> net.core.rmem_max = 16777216
> net.core.wmem_max = 16777216
> net.ipv4.tcp_rmem = 4096 87380 16777216
> net.ipv4.tcp_wmem = 4096 65536 16777216
> net.core.netdev_max_backlog = 250000
>
> What NIC are you using?  4Gb/s is very low for 10GbE.
>   
While I shouldn't specify the vendors at this point, with the default 
packet / socket sizes, an MTU of 1500 and playing with the affinity I 
can get this cardset over 7G before I run out of CPU. Higher with more 
tricks and netperf instances....
>
> Drew
>