[netperf-talk] FreeBSD sysctl cpu utilization diffs

[Andrew Gallatin]

Rick Jones writes:

 > Well, the appearance of 32 CPUs.  How one handles "CPU utilization" in 
 > the context of HW threading is a very "interesting" topic - one I'm 
 > running into for AIX with netperf4.

Even what "CPU utilization" means on a hyperthreaded P4 on Linux is
confusing...  If vmstat says "50%", that doesn't really mean 
you could really get twice as much work out of the box.
I also imagine the looper would greatly impact performance on a
hyperthreaded system.

 > > Looper does not seem to compile in 2.4.1, so it is hard to say. :(
 > 
 > I think that is resolved in the SVN top of trunk:

Excellent.  I'll try the SVN version soonish.

 > 
 > http://www.netperf.org/svn/netperf2/trunk/

 > I'll go ahead and apply your patch to the TOT, based on your assertion 
 > of the pickiness of the FreeBSD timekeepers :)  Working calibration-free 
 > mechanisms in netperf2 is a good thing - makes it easier to do the same 
 > in netperf4.

Great.  Thank you!

 > > BTW, I *love* the CPU binding for linux in 2.4!
 > 
 > I finally bit the bullet and agreed to accept CPU binding for non-looper 
 > processes - it should be there for most platforms now.  It is also in 
 > the netperf4 glib_migration branch.  A bit more sophisiticated now - we 
 > launch test threads from the same CPU to which they are to be bound 
 > (allow stack allocation to be based on the CPU to which the test thread 
 > will be bound)

It makes for really interesting performance data.  Receiving a
TCP_SENDFILE via 10GbE on a 2.0GHz Athlon64x2 running RHEL4 shows
8.4Gb/s at 50% CPU when netserver and the interrupt are both bound the
the same core.  If I move the netserver to the other core, bandwidth
shoots up to 9.4Gb/s at ~60% CPU.  Left to its own devices, linux will
schedule the netserver on the same core as the interrupt handler ran
on, so I was used to seeing the lower number.  Solaris uses one core
for the interrupt handler, and the other for the netserver and manages
9.8Gb/s at a claimed 48%CPU.  It will be fun to try 2.4.1 on Solaris
and see what happens when I bind netserver to the same core as the
ithread.

Sorry for rambling.  All these new features make me feel like a kid
on his birthday ;)

BTW, I had to beat on the CPU binding code to get it to choose 
the correct version on RHEL4.  I'll see if its still broken in
SVN and submit a patch if it is.

Thanks for such a wonderful tool!

Drew