traffic control

Hi all,

I'm making a nat box which will be shared among a few people wirelessly,
from remote locations via PPTP+MPPE. It works quite good but now i'd
like to add a new feature to it: priorities.

As the bandwidth is limited (512kbit ADSL), I'd like to give some users
priority over others. As i have several PPP interfaces (because pptp), I
had the idea of limiting everyone through an IMQ device. Two actually,
one for server->clients and one for clients->server.

So I patch my kernel and stuff and add a root HTB qdisc to the IMQ, then
a root class, and two leaves: one rate 100/ceil 500kbit and the other
rate 400 ceil 500. The default class is the 100/500 and selected IP
addresses go to the 500/500 class via a tc filter. As I want to limit
the speed of internal IP addresses I put the iptables -j IMQ to POSTROUTING.

Then I started making tests. On the external interface of the box I put
a webserver and a 600MB file. I start downloading on the low priority
box. It instantly gets 500kbit. Then I start downloading the same file
on the high priority box, and it gets the file at 400K wile the other
quickly throttles to 100kbit.

Finally, I try to do that on the real thing, and try to get the J2SDK
from java.sun.com. Low priority gets the file at 500kbit and I start
downloading on the higher priority box. It gets the file at 200kbit, and
just stays there. Doesn't pick up even after 10 minutes. So my question
is, am I just doing something wrong or it's just impossible to shape
incoming internet traffic?

hjf

--
Sí esta atascado, fuércelo. Sí se rompe, es que necesitaba ser reemplazado.

http://www.hjf.com.ar/

In comp.os.linux.networking Hern?n Freschi <(E-Mail Removed)>:
> Hi all,

> I'm making a nat box which will be shared among a few people wirelessly,
> from remote locations via PPTP+MPPE. It works quite good but now i'd
> like to add a new feature to it: priorities.

[..]

> Finally, I try to do that on the real thing, and try to get the J2SDK
> from java.sun.com. Low priority gets the file at 500kbit and I start
> downloading on the higher priority box. It gets the file at 200kbit, and
> just stays there. Doesn't pick up even after 10 minutes. So my question
> is, am I just doing something wrong or it's just impossible to shape
> incoming internet traffic?


AFAIK not with tc. IIRC the lartc howto says so. With p2p you can
reasonable use tc to shape outgoing to your needs, preventing the
system from hogging your whole bandwidth (in+out). For http/ftp
setting up squid transparent (http://www.squid-cache.org/) and
looking into it's delay classes features should be more
promising.

Good luck

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 189: SCSI's too wide.

> AFAIK not with tc. IIRC the lartc howto says so. With p2p you can
> reasonable use tc to shape outgoing to your needs, preventing the
> system from hogging your whole bandwidth (in+out).

Yes, I read on the lartc howto that outgoing data can be shaped and
incoming data can be "policed", but that the IMQ driver could be used to
shape incoming data. Anyway what I *think* is happening is this:

The IMQ is shaping data, that is, it can just delay what it's being fed.
When I was testing my setup, I did it with a local webserver at lan
speed. My root qdisc had a rate of 500kbit, but my webserver was bombing
it at about 100Mbit. When packets are going through the filter, HTB sees
that packets for higher priority clients start to get queued, while
lower priority clients are just fine. So, HTB starts delaying packets
for lower priority clients. So the high priority clients get dequeued
and get a higher rate.
BUT when I try that in "real life", and a low priority client is already
at ceil rate, all HTB sees are packets coming in for high priority, but
these packets don't get enqueued because they're coming in slowly, and
hence the low priority queue is not touched. I guess then that the
solution would be to use 300kbit out of a 1Mbit link and that way
things will work, because the queues will start getting full. But of
course that doesn't make any sense, buying 1Mbit just to use 300k!
I wonder if I use 400k out of 500k, then the traffic bursts in the
excess 100k would make the queues start filling?

hjf

--
Sí esta atascado, fuércelo. Sí se rompe, es que necesitaba ser reemplazado.

http://www.hjf.com.ar/

Hernán Freschi wrote:
>
>> AFAIK not with tc. IIRC the lartc howto says so. With p2p you can
>> reasonable use tc to shape outgoing to your needs, preventing the
>> system from hogging your whole bandwidth (in+out).
>
>
> Yes, I read on the lartc howto that outgoing data can be shaped and
> incoming data can be "policed", but that the IMQ driver could be used to
> shape incoming data. Anyway what I *think* is happening is this:
>
> The IMQ is shaping data, that is, it can just delay what it's being fed.
> When I was testing my setup, I did it with a local webserver at lan
> speed. My root qdisc had a rate of 500kbit, but my webserver was bombing
> it at about 100Mbit. When packets are going through the filter, HTB sees
> that packets for higher priority clients start to get queued, while
> lower priority clients are just fine. So, HTB starts delaying packets
> for lower priority clients. So the high priority clients get dequeued
> and get a higher rate.
> BUT when I try that in "real life", and a low priority client is already
> at ceil rate, all HTB sees are packets coming in for high priority, but
> these packets don't get enqueued because they're coming in slowly, and
> hence the low priority queue is not touched. I guess then that the
> solution would be to use 300kbit out of a 1Mbit link and that way
> things will work, because the queues will start getting full. But of
> course that doesn't make any sense, buying 1Mbit just to use 300k!
> I wonder if I use 400k out of 500k, then the traffic bursts in the
> excess 100k would make the queues start filling?

It should work OK at 400/500. If you haven't done so already maybe
limiting queue length by putting bfifos on htb leafs will cause drops
and help.

I don't get the bit about 300k/1meg - you need to back off from incoming
rate to ever build up a queue - for outgoing you back off to allow for
overheads. If you know enough about your DSL overheads you can tweak
things so you don't need to back off much on outbound.

http://www.adsl-optimizer.dk/

Andy.

> I don't get the bit about 300k/1meg -
Never mind, that was just an exaggeration to make a point.

> you need to back off from incoming
> rate to ever build up a queue - for outgoing you back off to allow for
> overheads. If you know enough about your DSL overheads you can tweak
> things so you don't need to back off much on outbound.

Well this box wont be used on a regular dsl (I'm just testing it on
mine), but on a SLA'd HDSL plugged quite near the backbone of the major
Telecom provider on my country. It's a 1Mbit connection which gives
about 133KB/s sustained 24/7.


hjf

--
Si esta atascado, fuércelo. Si se rompe, es que necesitaba ser reemplazado.

http://www.hjf.com.ar/

> It should work OK at 400/500. If you haven't done so already maybe
> limiting queue length by putting bfifos on htb leafs will cause drops
> and help.
Well I did some testing and got a 400k steady from my line so I set the
max to 350kbit and divided between 280-350 and 70-350. Then I fiddled
with a pfifo (which I found easier to manipulate) and set it to 20
packets for the high priority queue and 4 for the lower one. So when
data starts arriving through my extra 50k, it gets in a long queue for
the high priority clients, so the sender won't see the taildrop and
start slowing down. OTOH, as the low priority queue is low, packets are
dropped quickly when the network starts getting congested. Is my
reasoning correct?

Anyway it seems to work When high-prio starts downloading a file, low
prio's download slows down quickly, as expected.

Also I use a 350kbit root class and two 350kbit ceil children. My max
rate (the line rate) is 400kbit. Should I set the root class to 350 or
400, while the ceil of both children is 350?


hjf

--
Sí esta atascado, fuércelo. Sí se rompe, es que necesitaba ser reemplazado.

http://www.hjf.com.ar/

Hernán Freschi wrote:
>> It should work OK at 400/500. If you haven't done so already maybe
>> limiting queue length by putting bfifos on htb leafs will cause drops
>> and help.
>
> Well I did some testing and got a 400k steady from my line so I set the
> max to 350kbit and divided between 280-350 and 70-350. Then I fiddled
> with a pfifo (which I found easier to manipulate) and set it to 20
> packets for the high priority queue and 4 for the lower one. So when
> data starts arriving through my extra 50k, it gets in a long queue for
> the high priority clients, so the sender won't see the taildrop and
> start slowing down. OTOH, as the low priority queue is low, packets are
> dropped quickly when the network starts getting congested. Is my
> reasoning correct?

Well - I guess it will help doing it like that. I was thinking of the
same lemgth bfifo just because htb defaults can be very long. b rather
than pfifo so you don't drop small packets when you don't need to.

Adding the queues may not be what made it work anyway - finding the
correct rate is the most important thing.

400kbit for a 512 seems a bit low - was that at IP level or data - HTB
uses IP throughput for rates.

>
> Anyway it seems to work When high-prio starts downloading a file, low
> prio's download slows down quickly, as expected.
>
> Also I use a 350kbit root class and two 350kbit ceil children. My max
> rate (the line rate) is 400kbit. Should I set the root class to 350 or
> 400, while the ceil of both children is 350?

If you really can only get 400kbit without shaping then use 350 if
that's what works. There are lots of variables in reality and you can
see different results depending on the tcp options/type of client and
server.

For just bandwidth allocation without trying to shape for low latency
you can push the limits a bit closer. Shaping from the wrong end of the
bottleneck is not ideal - in your case the length of the buffer on the
other end will make a difference, if it's short then you are more likely
to see drops that are out of your control.

Andy.

Andy Furniss wrote:
> Well - I guess it will help doing it like that. I was thinking of the
> same lemgth bfifo just because htb defaults can be very long. b rather
> than pfifo so you don't drop small packets when you don't need to.

mm yes, I thought of that later, cause most of the use will be web
surfing with all those little images and short pages. i'll have to do
some more testing.

> Adding the queues may not be what made it work anyway - finding the
> correct rate is the most important thing.
No, I didn't mean to say that. What I meant was that the queues may help
me to redistribute the bandwidth faster, but that I'll have to check.
>
> 400kbit for a 512 seems a bit low - was that at IP level or data - HTB
> uses IP throughput for rates.
>
400kbit was measured with a Windows client downloading, as reported with
tc -s class show. Anyway 400k is a good number because I'm not in the US
and the DSL provider here (Argentina) is a monopoly...

hjf

--
Sí esta atascado, fuércelo. Sí se rompe, es que necesitaba ser reemplazado.

http://www.hjf.com.ar/

Hernán Freschi wrote:

> 400kbit was measured with a Windows client downloading, as reported with
> tc -s class show. Anyway 400k is a good number because I'm not in the US
> and the DSL provider here (Argentina) is a monopoly...

Ahh yes contention - a moving target - tricky :-)

I also have a monopoly teleco in the UK but though they sold my 512 dsl
as 50:1 it isn't really - people do see contention especially on higher
speed connections and oversubscribed exchanges.

FWIW HTB uses a very long average for its stats about 100sec to report
true rate for LAN speed tests I've done - so be sure to do a long test.

Andy.