[netperf-talk] How is throughput and round-trip time measured..

[Rick Jones]

On Wed, 2011-04-20 at 11:30 -0400, Priya wrote:
> Dear all,
> 
> 
> Sometime ago I started developing benchmark tests to verify the
> performance of a custom networking application, and I am comparing the
> results of my benchmark tests with those obtained from netperf.
> 
> 
> In order to make sure that the comparisons are sane, I have to
> understand exactly what goes inside the code of the netperf
> benchmarks. For example, I have misgivings about the correct way in
> which "average through-put" should be computed. Say we have three
> samples (observations) about data transmission over the network in
> terms of bit sent in number of seconds:
>      1. 5 bits in 1 sec
>      2. 15 bits in 2 sec
>      3. 10 bits in 3 sec
> In this case, will the average throughput be computed as a ratio of
> total bits over total time ( (5+15+10) / (1+2+3)  = 30/6 = 5 bits /
> sec). Or would it be computed as a average of the three throughputs
> i.e., (  ((5/1) + (15/2) + (10/3)) / 3 ) = 5.27 bits /sec) ? 
> 
> 
> While the first method is what is generally taught in class, the
> second method is what falls more in line with the notion of average =
> (total of all the observations / number of observations).
> 
> 
> I want to know how these issues are handled by the netperf code, for
> example while reporting the throughput in the TCP_STEAM test. 
> 
> 
> While the netperf documentation provides some pointers to this, it
> does not give me all the necessary details. Please guide me regarding
> how to go about understanding the theory behind the tests.

A standard TCP_STREAM test will track how many total bytes it sent over
the length of the test and divide by the elapsed time to arrive at the
average thoughput over the entire test.  It is not sampling part way
through the test.  So, if over a 60 second test netperf transferred 60
GB it would report 1 GB/s (if asked to use those units :)

You can confirm this by examination of the source if you wish.

With that out of the way, there is the "demo mode" (./configure
--enable-demo) where one can ask netperf to emit results over a
specified interval during the test.  It does those just as if they were
little "micro tests" but those interim results are not used to compute
the final result.  It remains computed as before.

Now, as to round-trip time.  For a stock netperf TCP_RR test, the method
is quite similar - transactions completed divided by the test length to
arrive at transactions per second.  As a stock netperf TCP_RR test has
only one transaction in flight at one time, one can, and netperf does,
invert that to arrive at seconds (microseconds) per transaction.

happy benchmarking,

rick jones