[netperf-talk] MacOSX vs Linux

[Andrew Gallatin]

When running some benchmarks with a MacOSX 10.5.5 sender and a
Linux 2.6.27.4 receiver, I noticed that the elapsed time was
always 3 seconds greater than it should be.  Notice how
the elapsed time is 4.00 seconds for a 1 second test:

% netperf244 -H172.31.193.15 -l 1 -P 0
  87380 131070 131070    4.00      236.97


This 3 seconds is driving me nuts, as it changes all the
benchmark results.  It doesn't seem to matter what NIC
is used, (10Gbe, 1Gbe, 100Mb).

According to dtruss on the Mac, it is taking 3 seconds for the
recvfrom() to finish:

<....>
    1005 sendto(0x4, 0x2E000, 0x1FFFE)            = 131070 0
     461 sendto(0x4, 0x4F000, 0x1FFFE)            = 41870 0
       4 sigreturn(0xBFFFED1C, 0x1E, 0x1FFFE)             = 0 Err#-2
      10 shutdown(0x4, 0x1, 0x1FFFE)              = 0 0
2997320 recvfrom(0x4, 0x4F000, 0x1FFFE)          = 0 0
      13 close(0x4)               = 0 0
<....>

I don't see these problems either going from linux <-> linux
or from MacOSX to MacOSX.  Just MacOSX -> Linux.


A tcpdump taken from the linux side at the very end of the
connection shows roughly what is happening.  The Mac is
172.31.193.22 and the linux box is 172.31.193.15:

13:27:55.284990 IP 172.31.193.22.49197 > 172.31.193.15.38471: . 
11930621:11932081(1460) ack 1 win 65535
13:27:55.285004 IP 172.31.193.15.38471 > 172.31.193.22.49197: . ack 
11932081 win 10390
13:27:55.285070 IP 172.31.193.22.49197 > 172.31.193.15.38471: P 
11932081:11933071(990) ack 1 win 65535
13:27:55.285072 IP 172.31.193.22.49197 > 172.31.193.15.38471: F 
11933071:11933071(0) ack 1 win 65535
13:27:55.285092 IP 172.31.193.15.38471 > 172.31.193.22.49197: . ack 
11933071 win 10395
13:27:55.285116 IP 172.31.193.15.38471 > 172.31.193.22.49197: F 1:1(0) 
ack 11933072 win 10395
13:27:55.285143 IP 172.31.193.15.12865 > 172.31.193.22.49196: P 
257:513(256) ack 257 win 14
13:27:55.285323 IP 172.31.193.22.49196 > 172.31.193.15.12865: . ack 513 
win 65535
13:27:58.284048 IP 172.31.193.15.38471 > 172.31.193.22.49197: F 1:1(0) 
ack 11933072 win 10395
13:27:58.284227 IP 172.31.193.22.49197 > 172.31.193.15.38471: . ack 2 
win 65535
13:27:58.284562 IP 172.31.193.22.49196 > 172.31.193.15.12865: F 
257:257(0) ack 513 win 65535
13:27:58.284727 IP 172.31.193.15.12865 > 172.31.193.22.49196: F 
513:513(0) ack 258 win 14
13:27:58.488004 IP 172.31.193.15.12865 > 172.31.193.22.49196: F 
513:513(0) ack 258 win 14
13:27:58.488087 IP 172.31.193.22.49196 > 172.31.193.15.12865: . ack 514 
win 65535


Does anybody have any ideas why netperf is "lingering" for 3 seconds?

Thanks,

Drew