[netperf-talk] SCTP Benchmark significantly slow

[Rick Jones]

Vlad Yasevich wrote:
> Thanks to rick for forwarding this.

Thanks Vlad for replying.  Since Vlad may not be subscribed to 
netperf-talk, I'll send this reply with all his text so it will enter 
the archives :)

rick jones

> 
> Rick Jones wrote:
> 
>>Chris Lydick wrote:
>>
>>>The only issue is that for some reason, I am unable to get any "decent"
>>>results using the SCTP_STREAM test within Netperf. Here are some of my
>>>findings:
>>>
>>>Using a 1Gbps Ethernet line between two nodes, I receive nearly
>>>987Mb/sec through the TCP_STREAM (this seems very accurate),
>>
>>If you have JumboFrames enabled at least.
>>
>>
>>>while on
>>>the SCTP_STREAM side of things, I am only receiving at most 2.4 Mb/sec.
>>>
> 
> 
> That's close to what I can get.  I can squeeze out ~3mb/sec with the latest
> kernel sources.
> 
> SCTP is frightfully inefficient and has not gone through a full performance
> analysis.  The biggest roadblock is the use memory copies.  There are at
> least 2 memory copies for every data message.  If the message is bigger then
> the mtu, there will be more memory copies.
> 
> Second, the receive buffer management is crufty and prevents us from fully utilizing
> our congestion window.  This is being addressed though.
> 
> There are other areas as well.
> 
> 
>>>I notice similar behavior when using one machine as both the source and
>>>destination of the test.
>>>
> 
> 
> You should get better results over loopback since we don't do checksumming there.
> 
> 
>>>I am using two Linux (Ubuntu 7.04) machines for the benchmarking,
>>>equipped with libsctp1 (1.0.6.dfsg-4), and libsctp-dev (1.0.6.dfsg-4)
>>>libraries.
> 
> 
> This is fine, but the kernel might not have all the latest bits.  Don't know
> how well Ubuntu tracks upstream patches.
> 
> 
>>>I have also tried adjusting the buffers through the command line
>>>prompts, but they have had little effect.
>>>
>>>Do you have any suggestions?
> 
> 
> Max out the buffers in the kernel ([r/w]mem_default and [r/w]mem_max).
> The _max thresholds limit what the user can specify.
> 
> 
>>Some additional specifics wrt parameters being passed on the netperf
>>command line would I suspect help :)
>>
>>Other things which are often useful include:
>>
>>*) CPU utilization, while netperf doesn't report it, per-CPU util might
>>be nice too if this is between MP systems.  Most GbE NICs can do CKO and
>>TSO, which will help TCP but will do zip for SCTP.
>>
>>*) Checking ethtool stats for losses
>>
>>*) Checking netstat stats for losses (I'm ass-u-me-ing there are sctp
>>stats)
> 
> 
> What Rick said.
> 
> -vlad