[netperf-talk] A bounced question about confidence intervals

[Rick Jones]

I saw the following as a bounced email - perhaps from an unsbuscribed 
person.  Anyway the question was interesting enough that I thought I 
would forward it and provide what answers I could:

> Hello,
> What are the "confidence level" and "interval" (-I option)? I find it
> has to do with the max and min number of iterations (-i option)... An
> iteration refers to a performance of the test? How does the system
> choose how many times the test must run based on this two numbers? I
> don't understand what are they used for from what I read and they are
> used in the example scripts.
> Thanks,
> Silvia

It is all a question of statistics.  If one just runs netperf once, one 
cannot be sure that the result wasn't an anomaly.  The -I and -i options 
are there to address that.

The confidence level is how statistically "confident" (ie certain) is it 
that the result being reported is within +/- N% of the "real" average. 
So, for example -I 99,5 asks netperf to be confident that 99 times out 
of 100, the result it is reporting is within +/- 2.5% of the "real" result.

The iterations control how many times netperf will run the specified 
test to try to achieve that level of "confidence." For example, -i 10,3 
tells netperf to run at least 3 iterations of the test, but no more than 
10.  If the confidence levels are achived within 10 iterations, all is 
well.  If they are not, netperf emits a warning giving the +/- that was 
achieved.

hth,

rick jones