Marked packets causing problems with internal route.

I recency posted a problem on sharing 2 internet connections for different LANs.
I would like to thank the guy (P Gentry) who provided me the link for this.

Good news is, I finally got it working.
I have the following setup: http://accessdenied.darktech.org/2.txt

Bad news is, I now have problems with internal routing (company A & B senario routing).

My setup:
eth0 and eth1 (coy A/B) are connected to my Local area network:

eth0=10.30.1.0
eth1=10.20.0.0

eth2 is connected to the internet via a lease line.
ppp0 is connected to an ADSL provider:

eth2=161.18.195.4
ppp0=220.255.206.242

So to let users on eth0 tap on internet via eth2
and
the users on eth1 tap on the internet via ppp0(adsl), I do this:

#Start sharing internet from 2 providers

#Enable NAT routing from ppp0 (default route) for eth1 (coy B)
iptables -t nat -A POSTROUTING -o ppp0 -j SNAT -s 10.20.0.0/22 --to-source 220.255.206.242

#Enable NAT routing from eth2 for eth0 (coy A)
iptables -t mangle -A PREROUTING -s 10.30.1.0/24 -j MARK --set-mark 5 #This line causes the
problem.
iptables -t nat -A POSTROUTING -o eth2 -j SNAT -s 10.30.1.0/24 --to-source 61.8.195.4

#Add the route/marker for the eth0/eth2
/sbin/ip route flush table 5
/sbin/ip route add table 5 default via 161.18.195.1
/sbin/ip rule add fwmark 5 table 5

#End sharing internet from 2 providers

That shares the internet for the 2 networks (coy A & B).

Now, because of this line:
iptables -t mangle -A PREROUTING -s 10.30.1.0/24 -j MARK --set-mark 5

It screws up routing btw eth0 and eth1. Remove that line, and problem disappears (obviously
coy A/eth0 can't access the internet then).

It seems that all packets coming to eth0 are been marked '5'. So packets coming from eth1
(10.20.0.0)->eth0 also end up being marked I suppose? And therefore end up on the wrong gateway
(161.18.195.1)

I solved the problem by inserted another marker.

iptables -t mangle -A PREROUTING -s 10.30.1.0/24 -d 10.20.0.0/22 -j MARK --set-mark 6
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 10.20.1.1

#Add the route/marker for the eth0/eth1
/sbin/ip route flush table 6
/sbin/ip route add table 6 default via 10.20.1.1
/sbin/ip rule add fwmark 6 table 6

Ok everything works now but is there a easier or more appropriate way to do this?
Am I doing this right?
I don't want to do this all the time when I include new networks.

Thanks





To e-mail, remove the obvious

AcCeSsDeNiEd <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>. ..
[snip]>
> Now, because of this line:
> iptables -t mangle -A PREROUTING -s 10.30.1.0/24 -j MARK --set-mark 5
>
> It screws up routing btw eth0 and eth1. Remove that line, and problem disappears (obviously
> coy A/eth0 can't access the internet then).
>
> It seems that all packets coming to eth0 are been marked '5'. So packets coming from eth1
> (10.20.0.0)->eth0 also end up being marked I suppose? And therefore end up on the wrong gateway
> (161.18.195.1)

Ummm ... don't have time to think about this now -- see below for ref.

> I solved the problem by inserted another marker.
>
> iptables -t mangle -A PREROUTING -s 10.30.1.0/24 -d 10.20.0.0/22 -j MARK --set-mark 6
> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 10.20.1.1
>
> #Add the route/marker for the eth0/eth1
> /sbin/ip route flush table 6
> /sbin/ip route add table 6 default via 10.20.1.1
> /sbin/ip rule add fwmark 6 table 6
>
> Ok everything works now but is there a easier or more appropriate way to do this?
> Am I doing this right?
> I don't want to do this all the time when I include new networks.
>
> Thanks

OK, you have set up some additional tables but where are your rules in
the RPDB that select/direct packets through those tables? Check here
about rules (as well as lartc and Guide to IP Layer ...):
http://www.linuxgrill.com/iproute2.doc.html#ss9.6

When I have more time I'll look at this and get back.

hth,
prg
email above disabled

Hi thanks for the help again man.

On 20 Aug 2004 18:28:52 -0700, (E-Mail Removed) (P Gentry) wrote:


>OK, you have set up some additional tables but where are your rules in
>the RPDB that select/direct packets through those tables?


You meant these lines? :

/sbin/ip rule add fwmark 5 table 5
/sbin/ip rule add fwmark 6 table 6

Yup they are there. And it is working but I was wondering if there is an alternative/easier
solution?

Thanks

To e-mail, remove the obvious

AcCeSsDeNiEd <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>. ..
> Hi thanks for the help again man.
>
> On 20 Aug 2004 18:28:52 -0700, (E-Mail Removed) (P Gentry) wrote:
>
> >OK, you have set up some additional tables but where are your rules in
> >the RPDB that select/direct packets through those tables?
>
> You meant these lines? :
>
> /sbin/ip rule add fwmark 5 table 5
> /sbin/ip rule add fwmark 6 table 6
>
> Yup they are there. And it is working but I was wondering if there is an alternative/easier
> solution?
>
> Thanks

I think you need to use a "better" selector in your rule -- think
above will add "from all" when what you want is "from src-addr"
(subnet/prefix) and perhaps "iif ethx" and maybe some others. In
fact, now that I look more closely (and think about what I'm looking
at) you _may_ not need to use FWMARK -- just straight source routing.

Eg., (adapted from http://lartc.org/howto/lartc.rpdb.html )
[root]# ip rule add from 10.30.1.0/24 table 5
[root]# ip rule show
0: from all lookup local
32765: from 10.30.1.0/24 lookup 5 << first rule that matches "wins"
32766: from all lookup main
32767: from all lookup default

Table 5 would provide the proper GW/default route. You could add
another table (6) and rule similarly for the other subnet and its GW.

BTW, it's best to use this to view rule look-ups:
[pbrain]$ /sbin/ip rule show << list the entire RPDB
0: from all lookup local << these are the default
32766: from all lookup main << prios, tables and rules
32767: from all lookup 253

[Gotcha avoidance]
Note that iptables uses decimal notation for fwmark -- ip uses hex:
http://mailman.ds9a.nl/pipermail/lar...q3/005039.html

This is the part I always had to fiddle with -- the syntax is
deceptively simple and is very easy to get "wrong" for what you want
to achieve. Basic idea is to make the selector as specific as
possible (but no more) and direct it to the proper table -- where the
actual routing/forwarding takes place.

Get this part working in a simplified fashion and you can explore the
need for setting PRIO so that the rules are "ordered" for optimized
look-ups. There are other goodies also.

I'm just getting the hang of this stuff after a "real" attempt to
understand it rather than just playing with it. Your solution works,
I think, by routing packets through the system twice before forwarding
onto the nic. The harry part is understanding how ip(tools) and
iptables interact -- excepting traffic control of course ;-)

Unfortunately, I've not found a good source/ref regarding rules --
just a few simplified examples. The best discussion is from the guy
that wrote the code:
http://www.policyrouting.org/iproute2.doc.html#ss9.6
Required reading ;-)

This can be useful also (from same site):
http://www.policyrouting.org/PolicyR...NLINE/TOC.html

And more rules examples:
http://linux-ip.net/html/tools-ip-rule.html

This diagram may/may not be handy:
http://www.docum.org/docum.org/kptd/

hth,
prg
email above disabled