[netperf-talk] Any stacks out there willing to tell me about queued but unsent data?

[Cook, Jonathan]

I don't think there is a universal way of doing this.  The standard MII/GMII registers do not tell you if there is data waiting to be sent.  If you are using BCM5714 or BCM5715 MAC you can query the UMP Status Register to see if the TX FIFO is empty.  Other hardware devices may have different registers.  Broadcom's description of the TX FIFO explains their philosophy: "The TX FIFO provides elasticity, while data is moved from device internal memory to PHY. There are no programmable settings for the TX FIFO. This FIFO's operation is completely transparent to host software."  So you're not supposed to worry about the FIFO.

Jon Cook

-----Original Message-----
From: netperf-talk-bounces at netperf.org [mailto:netperf-talk-bounces at netperf.org] On Behalf Of Rick Jones
Sent: Monday, July 30, 2012 7:41 PM
To: netperf-talk at netperf.org; netperf-dev at netperf.org
Subject: [netperf-talk] Any stacks out there willing to tell me about queued but unsent data?

I'm wrestling with something in netperf and could use some help.  I try to have my cake, eat my cake and perhaps even have someone else pay for the cake when it comes to measuring maximum packets per second with single-byte, burst-mode netperf TCP_RR tests.  I ass-u-me that if I set TCP_NODELAY, and am careful to walk-up the number of outstanding transactions at one time on the connection (to deal with an initial congestion window) then I can take my transactions per second and multiply it by two to get packets per second (when my request and response sizes are < the MSS).

Now, if there are any TCP retransmissions, that assumption rather goes out the window.  If I am doing "by-hand" performance testing with a single system I can wrap my netperf commands with netstat easily enough and see if the assumptions hold.  I can even wrap the netperf command with some link-level statistics gathering and further check things.

However, when I am running large-scale, automated tests that starts to become impractical.  Particularly if I'm talking about say hundreds of simultaneous netperf instances.  In that case I rather need each netperf to have some way to ascertain that bundling is taking or may have taken place.

Under Linux I can see if there have been any retransmissions on the connection via a getsockopt(TCP_INFO).  I could then ass-u-me that if I checked that periodically (infrequently enough to keep the overhead low), and noticed the total retransmission counter increasing that I've had retransmissions and restart my own "slow-start" but what I think would be more helpful/immediate would be some way to know if there is data queued to the connection but not yet sent.  Are there any stacks out there which can do that with some sort of getsockopt() or ioctl()?

I'm sure there is nothing "universal" out there, particularly since I'm cheating about a fundamental aspect of TCP here - that it is but a byte-stream.  But if there is a way to get closer I'd love to hear about it.

happy benchmarking,

rick jones

_______________________________________________
netperf-talk mailing list
netperf-talk at netperf.org
http://www.netperf.org/cgi-bin/mailman/listinfo/netperf-talk