[netperf-talk] MacOSX vs Linux

[Rick Jones]

I think netperf is stuck waiting for the Mac to accept Linux's FIN. 
I've stripped the packet trace closer to its essence, replacing IPs with 
names, removing redundant time information, and packets for the control 
connection and I see:

::55.284990 IP Mac.49197 > Linux.38471: . 11930621:11932081(1460) ack 1 
win 65535
::55.285004 IP Linux.38471 > Mac.49197: . ack 11932081 win 10390
::55.285070 IP Mac.49197 > Linux.38471: P 11932081:11933071(990) ack 1 
win 65535
::55.285072 IP Mac.49197 > Linux.38471: F 11933071:11933071(0) ack 1 win 
65535
::55.285092 IP Linux.38471 > Mac.49197: . ack 11933071 win 10395
::55.285116 IP Linux.38471 > Mac.49197: F 1:1(0) ack 11933072 win 10395
::58.284048 IP Linux.38471 > Mac.49197: F 1:1(0) ack 11933072 win 10395
::58.284227 IP Mac.49197 > Linux.38471: . ack 2 win 65535

Notice how the Mac is not ACKing the first FIN from Linux at 55.28. 
That suggests the Mac 10.5.5 stack didn't like the FIN for some reason 
and dropped it.  Since the FIN has been dropped, netperf will remain in 
the recv() call (gets mapped to a recvfrom() I suspect) waiting for the 
confirmation of the remote's receipt of data - which is carried in that 
recv() return of zero.

Only the retranmsission, which happens to be 3 seconds later, gets an 
ACK.  Admittedly, I am ass-u-me-ing it is the retransmission of the FIN 
which is ACKed and not some massively delayed ACK from the Mac.

So, the question is why didn't the Mac like that first FIN?  Do the 
netstat stats show any checksum failures or other drops on the Mac which 
might correlate with that ostensibly ignored FIN?

Of course there is the other question of why did Linux wait three 
seconds to retransmit the FIN - my guess there would be that since the 
Linux side never sent any data it never was willing to "train" its RTO 
from the initial RTO and that was perhaps three seconds.  Frankly though 
I'd think that it should have "counted" the ACK of the SYN|ACK at the 
beginning...

So, depending on what you see in netstat and/or link-level stats we may 
have two questions.  The first, for Apple and why their stack didn't 
like the Linux FIN.  The second, for netdev and why Linux took three 
seconds to retransmit the FIN.

happy benchmarking,

rick jones

BTW, we see the Linux side having to retransmit the FIN on the control 
connection too - but that one happens quicker - I suspect since there 
was traffic both directions on the control connection.