[netperf-talk] Any stacks out there willing to tell me about queued but unsent data?

[Rick Jones]

On 08/01/2012 02:50 PM, Cook, Jonathan wrote:
> I don't think there is a universal way of doing this.  The standard
> MII/GMII registers do not tell you if there is data waiting to be
> sent.  If you are using BCM5714 or BCM5715 MAC you can query the UMP
> Status Register to see if the TX FIFO is empty.  Other hardware
> devices may have different registers.  Broadcom's description of the
> TX FIFO explains their philosophy: "The TX FIFO provides elasticity,
> while data is moved from device internal memory to PHY. There are no
> programmable settings for the TX FIFO. This FIFO's operation is
> completely transparent to host software."  So you're not supposed to
> worry about the FIFO.

I was thinking more up at layer 4 (eg TCP) rather than down in the 
driver.  The problem I'm trying to workaround is the bundling of 
requests or responses into segments when TCP has to retransmit - perhaps 
more accurately when TCP shrinks its congestion window and will be 
willing to have only so many segments in flight at one time (there is 
the wrinkle of cwnd as octets vs packets in there too as well).

I want to be able to "cheat" (being honest) and ass-u-me that my netperf 
TCP_RR burst-mode transactions per second is also a segments and packets 
per second count. So I'm more concerned about whether TCP has 
transmitted something more than I am concerned about whether that 
"transmission" has actually made it out the egress interface.

I also posted my query to comp.protocols.tcp-ip and was pointed at the 
TIOCOUTQ ioctl(), which at least under Linux will tell me how many bytes 
are queued to the socket but not yet sent (from the perspective of TCP). 
  However, it still isn't clear to me if I will be able to kludge my way 
to what I want.  My first, very simplistic, attempts at that (not 
checked-in) have not been all that successful.

The other ways to address it might include:

*) Don't use burst-mode.  When there is only one transaction in flight 
at one time, (and it is sub-MSS in size) one knows the correspondence to 
segment and packet count holds.  However, that means running hundreds or 
perhaps even thousands of simultaneous netperf and/or netserver 
instances, which in the past at least made it as much a context 
switching benchmark as a networking benchmark - that is why I added 
burst-mode in the first place.

*) Drive multiple connections from a single thread, still without burst 
mode.  To do that "right" means to include select/poll/whatnot and logic 
to make sure that all connections are serviced evenly.  Starts to drift 
from "simple benchmark."

*) Just use UDP_RR instead.  That still has the issue of a lost 
request/response not being recovered without some form of detection in 
place.

*) Continue to hand-wave and simply suggest folks do sanity checks.

happy benchmarking,

rick jones