Traffic Shaping for two masqueraded interfaces?

I am looking to alter my network to introduce some traffic shaping so I
can improve my own 'net usage and share spare bandwidth with others.

Currently I have a simple arrangement:
ADSL router + switch (with NAT/PAT & simple firewall)
-> Everything else
where everything else is currently my main box, a small server (used
for http(s), bittorrent and rsync services to the 'net, SMB locally and
some SSH tunnelling to another site), an extra box or two when I'm
playing, and sometimes the work's laptop.

I plan to add two extra NICs to the server and use it as a router as
well as everything else (I may give up on this and use a spare box as
the router instead, but I'd like to keep the number of machine running
to a minimum of I can) so I would have eth0 (to ADSL router), eth1
(trusted machines, accessing the ADSL via IP-Masq) and eth2 (other
machines, also IP-masqed). The reason to want eth1+eth2 and not put
everything on the same interface is that I'd like to put some firefall
rules to block most traffic between machines on the two interfaces (I
may drop this complication initially and rely on securing individual
machines, but I'd rather not).

I have a few questions about my plan though...

When shaping outgoing 'net traffic (eth0, anything going out to the
ADSL router): As well as basic QoS shaping (small packets and important
services have priority to improve latency, BitTorrent and such as low
priority as possible to stop it getting in the way of other stuff) I'd
like to put a seperatly classify some traffic for shaping from eth1,
eth2 and the router itself so the server and machines on eth1 get the
lion's share and those on eth2 get what is left. Am I right in
understanding that I can mark packets from eth1 and eth2 as the enter,
so that I can tell them apart when classifying the traffic in eth0's
egress qdisc(s) [i.e. any marks made using iptables directives will
persist for the life of the packet until that point]?

I'd like to try shape incoming traffic as well. I know this isn't
exactly possible as I have little control over what the 'net send in
but I can go some way by limiting how quickly traffic can be forwarded
on once it gts here. I would like to use HTB to, say, try guarantee the
machines on eth1 have 150Kb/s available and eth2 have 50 (with each
borrowing spare bandwidth allocation the others aren't using). If I
were to pass traffic for both eth1 and eth2 egress through IQM (to get
round the fact that normally shaping is interface specific) would I be
able to attach an HBT qdisc to that for this purpose? And would it's
filters be able to tell packets for the networks on eth1/eth2 apart
(i.e. would the packets have been 'un-masqueraded' before they are
classified in IQM/HTB?).

I know I can do the above more simply if I put all the masqueraded
machines on one interface and not split between eth1 and eth2 - but I'm
doing this as much to learn+play as any more practical purpose, so at
this point I'd rather try work out the complications instead of
dropping requirements to avoid them...

Dave wrote:
> Am I right in
> understanding that I can mark packets from eth1 and eth2 as the enter,
> so that I can tell them apart when classifying the traffic in eth0's
> egress qdisc(s) [i.e. any marks made using iptables directives will
> persist for the life of the packet until that point]?

Yes this should work.

>
> I'd like to try shape incoming traffic as well. I know this isn't
> exactly possible as I have little control over what the 'net send in
> but I can go some way by limiting how quickly traffic can be forwarded
> on once it gts here. I would like to use HTB to, say, try guarantee the
> machines on eth1 have 150Kb/s available and eth2 have 50 (with each
> borrowing spare bandwidth allocation the others aren't using). If I
> were to pass traffic for both eth1 and eth2 egress through IQM (to get
> round the fact that normally shaping is interface specific) would I be
> able to attach an HBT qdisc to that for this purpose? And would it's
> filters be able to tell packets for the networks on eth1/eth2 apart
> (i.e. would the packets have been 'un-masqueraded' before they are
> classified in IQM/HTB?).

Should be OK - if using 2.6 imq there are config options about where it
hooks in pre/postrouting (you can patch 2.4 imq to do the same). Select
"A" for prerouting then send traffic coming in on eth0 to IMQ from
prerouting mangle and it will be de-natted. You can use HTB OK and
traffic for the shaping box its self will be included aswell (will still
have real ip).

Andy.

Thanks for the reply Andy,

I'm going to leave IQM out of the equation for now, to save the extra
hassle of compiling it into the kernel and tools [IQM is not included
in the stock kernels for Debian for well documented (when you work out
where to look!) reasons], by dropping either the "all local traffic
shaped by one rule set" requirement or the "two local subnets seperated
by firewall" requirement [or both in turn - this is a learning exercise
after all!].

I can add IQM into the mix once I've got everything else working [and
understood!] in enough detail. Last time I played with kernel compiles
and network routing was back in the 2.0.x days [when a back-ported PPA
driver needed patching in for those new fangled Zip drive thingies, and
IP-MASQ was all new shiney and exciting!], so I've got some catching up
to do...

Another thought: as a "security through flimsy obscurity" approach I
caould always use aliaing give eth1 addresses on two sub-nets - if that
would work I could still firewall at the point of forwarding between
sub-nets.

Someone on subnet1 would only have to guess the IP range of subnet2 to
bypass the firewall rules, but there aren't going to be any determined
hackers to worry about [yet - though this would change if bits of the
network ever become more public, like when/if wireless becomes a factor
in the equation] - the main reason to seperate the two sets of machines
is to reduce the spread of damage caused if the "some tit infected with
a worm/virus/trojan/<insert-other-nasty>" problem rears its ugly head...