TCP congestion stats + performance tuning parameters

Hello:

I am doing some performance thruput measurements of our TCP stack by
using Linux 2.6.11 as a directly connected test client.

There seem to be some problem with congestion window leading to poor
thruput numbers.

Is there a way I can see (or infer) congestion window on linux side. I
see that the
receive window (advertised in the TCP pkt) is abt 32k but am not sure
if Linux is really
filling the pipe. Are there any counters/stats available for this
(netstat -s doesn't help much)?

Also, if someone knows about other tricks to tune up Linux for
performance, could you
please share the same. Any pointers...

Thanks for your time and help.

Regards,
Vishal.

vishal...@gmail.com wrote:
> Hello:
>
> I am doing some performance thruput measurements of our TCP stack by
> using Linux 2.6.11 as a directly connected test client.

What OS on "server"? Link speed?

> There seem to be some problem with congestion window leading to poor
> thruput numbers.

In what sense CW is the culprit? Small streams? Medium streams? Huge
streams?

> Is there a way I can see (or infer) congestion window on linux side. I
> see that the
> receive window (advertised in the TCP pkt) is abt 32k but am not sure
> if Linux is really
> filling the pipe. Are there any counters/stats available for this
> (netstat -s doesn't help much)?

You mean Linux is hitting CW limit when _sending_? CW used only by
sender (if smaller than advertised window size).

> Also, if someone knows about other tricks to tune up Linux for
> performance, could you
> please share the same. Any pointers...
>
> Thanks for your time and help.
>
> Regards,
> Vishal.

If you're testing the "other" box's stack implementation, I'm not sure
what you might want to change on the Linux box. The defaults are
pretty good for a _client_ but not so good as a high volumn server.
Anyway, it sounds like you would like Linux to suck up packets from the
other box as fast as possible, perhaps also shooting a stream at the
other box as fast as possible to see what the other box can put
out/take in.

Best place to start (since I've no real data points) is to look over
the networking params and make some changes. Look here:
http://ipsysctl-tutorial.frozentux.n...tml/index.html
http://ipsysctl-tutorial.frozentux.n...es.html#AEN329

You may have to read through the TCP variables several times to get a
sense which ones are most suitable for changing _and_ how some of the
variables interact. See especially:

3.3.14. tcp_mem
3.3.21. tcp_rmem
3.3.22. tcp_sack
3.3.29. tcp_window_scaling
3.3.30. tcp_wmem

You may also want to look at this:
http://www-didc.lbl.gov/TCP-tuning/TCP-tuning.html
http://www-didc.lbl.gov/TCP-tuning/linux.html

This last points out some 2.6 changes in TCP variables.

Chap. 20 of _TCP/IP_Illustrated_ if you have a copy around :-)

And if you want to try some Linux benchmark/perf tools... :

http://www.netperf.org/
http://dast.nlanr.net/Projects/Iperf/
http://ftp.arl.mil/~mike/ttcp.html
http://www.usinglinux.org/benchmarks/
http://ltp.sourceforge.net/tooltable.php

good luck,
prg

If you are using a GbE link you may want to try your tests with TSO
disabled (ethtool -k/-K) if it isn't already disabled.

rick jones
--
portable adj, code that compiles under more than one compiler
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Thanks guys...with your pointers, I was able to resolve the issue.


Rick Jones wrote:
> If you are using a GbE link you may want to try your tests with TSO
> disabled (ethtool -k/-K) if it isn't already disabled.
>
> rick jones
> --
> portable adj, code that compiles under more than one compiler
> these opinions are mine, all mine; HP might not want them anyway...
> feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...